October 2007Natural Language Processing1 CSA3050: Natural Language Algorithms Words and Finite State...

October 2007 Natural Language Processing 1

CSA3050: Natural Language Algorithms

Words and

Finite State Machinery


Acknowledgement

Material derived from/copied from– Jurafsky and Martin, Speech and Language

Processing, Prentice Hall 2000– Richard Sproat, Lecture notes


Outline

Words

Regular Languages

Regular Expressions

Finite State Automata


What is a Word?

• A series of speech sounds that symbolizes meaning without being divisible into smaller units

• Any segment of written or printed discourse ordinarily appearing between spaces or between a space and a punctuation mark

• A set of linguistic forms produced by combining a single base with various inflectional elements without change in the part-of-speech elements

• The smallest meaningful element of language. When written it stands alone with a space on either side of it.


Information Associated with Words

• Spelling– orthographic– phonological

• Syntax– POS– Valency

• Semantics– Meaning – Relationship to other words


Properties of Words

• Sequence– characters pollution– phonemes

• Delimitation– whitespace– other?

• Structure– simple ("atomic") words– complex ("molecular") words


Complex Words

• Complex words have subparts:• e.g. "enlargement"en + large + ment

• Some subparts are valid wordslarge

• Others are prefixes and suffixesen, ment

• N.B. The complex word can be built in different ways: (en + large) + menten + (large + ment)


Morphological Processes

• affixation– prefix– suffix– circumfix: għandi - mgħandix– infix: phenidine phenetidine

• other morphological processes– redoubling (mexa; mexxa)– vowel change (swim; swam)


Complex Words Formed by Concatenation

disreunen

largechargeinfectcodedecide

edingeeerly

+ +

prefixes roots suffixes


The Language of Words

• What kind of formal language is the language of words?

• One which can be constructed out of– A characteristic set of basic symbols (alphabet)– A characteristic set of combining operations

• Union (disjunction) • Concatenation• Iteration

• Regular Language; Regular Sets


Outline

Words

Regular Languages

Regular Expressions

Finite State Automota


Regular Languages

• A regular language is a language with a finite alphabet that can be constructed out of one or more of the following operations:– Set union– Concatenation– Transitive closure (Kleene star)


Some things that areregular languages

• Zero or more a’s followed by zero or more b’s

• The set of words in an English dictionary

• Dates

• URLs

• English?


Some things that are not regular languages

• Zero or more a’s followed by exactly the same number of b’s

• The set of all English palindromes (e.g. Madam I'm Adam)

• The set that includes all noun phrases of the form– the cat slept– the cat the dog bit slept– the cat the dog the man fed bit slept


Some special regular languages

• The universal language (Σ*)

• The empty language (Ø)

Note: the empty language is not the same as the empty string


Some closure propertiesof regular languages

• Intersection

• Complementation

• Difference

• Reversal

• Power


Characterising Classes of SetCLASS OF

SETS or LANGUAGES

NOTATION MACHINE


Outline

Words

Regular Languages

Regular Expressions

Finite Automota


Regular Expressions

• Notation for describing regular sets

• Used extensively in the Unix operating system (grep, sed, etc.) and also in some Microsoft products (Word)

• Xerox Finite State tools use a somewhat different notation, but similar function.


Regular Expressions

a a simple symbol

A B concatenation

A | B alternation operator

A & B intersection operator

A* Kleene star


Characterising Classes of Set

CLASS OFSETS or LANGUAGES

NOTATION MACHINE


Outline

Words

Regular Languages

Regular Expressions

Finite Automata


Finite Automaton• A finite automaton is a quintuple

(Q, I, q0,F, δ ) where:

• Q is a finite set of states

• Σ is alphabet of symbols

• q0 Q is a start state

• F Q are final states

• δ is a transition relation δ(q,i,q') between a state q Q, a symbol σ Σ and q' Q


Representation of FSA’s:State Diagram


State Table


Mr. Kleene


Kleene’s theorem

• Languages generated by NFAs are exactly equivalent to languages described by Regular Expressions.

• Kleene’s Theorem, part 1: To each regular expression there corresponds a NFA.

• Kleene’s Theorem, part 2: To each NFA there corresponds a regular expression.

http://www.cs.may.ie/~jpower/Courses/parsing/node6.html

http://www.cs.may.ie/~jpower/Courses/parsing/node6.html


Converting a Regular Expressionto an NFA

• The NFA representing the empty string is:

• The NFA representing a single character is:

1 2ε

1 2a


Regular Expression to NFA

Dia

gram

fro

m L

eoni

das

Feg

aras

, U

niv.

Tex

as


Deterministic Finite Automata

• In deterministic finite automata (DFA), every state/symbol pair maps to a unique state

• In other words, δ is a function• Why do we care about DFAs?


Deterministic Finite Automata

• In deterministic finite automata (DFA), every state/symbol pair maps to a unique state

• In other words, δ is a function• Why do we care about DFAs?• EFFICIENCY!!


Equivalence of NFA’s and DFA’s


Subset Construction for Determinisation

• States which are connected by an ε transition will be represented by the same states in the DFA.

• If there are multiple transitions based on the same symbol, then we can regard a transition as moving from a state to a set of states (ie. the union of all those states reachable by a transition on the current symbol).

• Thus these states will be combined into a single DFA state.

• more details http://www.cs.may.ie/~jpower/Courses/parsing/node9.html


Subset construction for determinization











Date post:	18-Jan-2016
Category:	Documents
Upload:	charlene-campbell
View:	220 times
Download:	0 times

October 2007Natural Language Processing1 CSA3050: Natural Language Algorithms Words and Finite State...

Documents