DH TOOLSIntroduction to Text Analysis
Cameron Buckner
Visiting Assistant Professor
Department of Philosophy
Our Initiative• Promote, facilitate, interact• Reading group• Speaker series• Infrastructure advocacy
• Tools workshops• Grantwriting support
http://www.uh.edu/class/digitalhumanities/
RoadmapGoal today: Analyze texts using cutting-edge analyses from computational psycholinguistics with an off-the-shelf tool, word2word
1. What can you do with text analysis?
2. A little bit of theory: Semantic spaces
3. BEAGLE: The holographic lexicon
4. MDS: Visualizing multidimensional networks
5. Examples
6. Hands-on play
What is DH?• Computation and interpretation• The use of computational tools for the
production, exploration, analysis, and dissemination of humanistic knowledge• Thread common between new and old:
pattern recognition
• Includes• Digitization and archiving, markup• Analysis & visualization• Search & dissemination• Pedagogy
Methods of Text Analysis I• Statistical analysis, information extraction, machine
learning• Syntactic: word frequencies (Google n-grams), vocabulary
usage, stylometry (authorship and genre), Pagerank
http://www.nytimes.com/interactive/2012/09/06/us/politics/convention-word-counts.html
Methods of Text Analysis II• Semantic: tf-idf, latent semantic analysis, latent dirichlet
allocation, entropy-based measures, ontologies• Aim to model relevance, semantic similarity, taxonomic
relationships, object properties and relations
Reminders• Be creative and have fun, but if you want to publish…• Be principled:• Junk in, junk out• Always know assumptions required by a method• Analyses should hold up under trivial transformations of data
representation
• Be prepared for pragmatic design decisions• Go in with hypotheses and structured questions• Confirm with careful humanistic interpretation
The Mental Lexicon• A “mental dictionary”• Contains information about:• Word meaning, grammatical roles, taxonomic relations, typical properties• Behavioral indicators: recognition speed, synonymy and relevance
judgments, priming, frequency effects, categorization
BEAGLE• A model that learns (unsupervised) a holographic mental
lexicon automatically from text• History: Two approaches to semantic analysis• Co-occurrence based measures (“bag of words”, LSA, tf-idf)• Good at determining relevance, bad at determining roles and
relations
• Order-based measures (n-gram models, generative grammars, hidden Markov models)• Good at identifying grammatical and structural relations, bad at
identifying relevance and meaning
• Challenge: Can the two be combined?
Context + Role• Assumption: People acquire an
idiosyncratic mental lexicon from patterns of co-occurrence and syntactic relationships they encounter in natural language.• “You shall know a word by both the
company it keeps and how it keeps it.”
• Goal: If we could build a representation of a text’s context/role distributions, we could predict the structure of a mental lexicon that produced a corpus and/or that would be produced by it• Texts as “mental fingerprints”
HowHologram
sWork
Basic Vector Approach1. Start with a multi-dimensional vector space
2. Each term meaning is initially represented by a random, constant environment vector and an empty memory vector
3. Associations between terms can be represented by adding or averaging their environment vectors into their memory vectors
4. Each time terms co-occur, their memory vectors become closer in multi-dimensional similarity space
Representing Order Info• Convolution: compressing outer-product matrix of two
term vectors so that the product contains recoverable information about both
• Example: z = x * y• Association vector z contains information about both x and y• Can (approximately) reconstruct source vector y by probing
z (deconvolution) with x (and vice versa)
• Combined BEAGLE memory vector: Context memory comes from vector addition, and order information comes from n-gram binding using convolution
Combined Memory Vector
• m = memory vector• e = initial random environment vector• p = position in sentence• lambda = constant chunking factor (size of n-gram window)• bind i,j = a non-commutative convolution of constant order vector
with other environment vectors in n-gram
Resonance retrieval…
So, BEAGLE method
1. Choose number of dimensions for vector space, size of n-gram window for order info
2. Clean up source documents using standard NLP (stop words, stemmers, etc.)
3. Learn context and order vectors from corpus, combine
4. Select words of interest
5. Visualize multi-dimensional space using favorite method (e.g. MDS)
Limitations of BEAGLE• Only considers 1-sentence windows• Lexical ambiguity• Valence (e.g. synonyms, antonyms)
MDS• A way to view a multi-dimensional similarity space• Collapses multi-dimensional space in way that tries to
mutually preserve distances between vectors• Collapsing dimensions often reveals most significant
[higher-order] dimensions
Uses• How do two academic reference works compare in their
coverage of a discipline?• Biases? Overlap?
InPhO-Semantics
Credit: Robert Rose
Black = SEP, Red = IEP
Credit: Jun Otsuka
Political rhetoric• What can we learn from the “semantic space” derived
from a party or candidate’s rhetoric?• Central issues?• Key comparisons?• Ideological focus/big tent?• Location on ideological spectrum?
• Example: compare speeches from Republican and Democratic political conventions
Heat Map: Terms most diagnostic of a speech’s being delivered by a Democrat“Hotter” indicates more diagnostic in comparison. Hottest terms = aarp, experience, affordable, abuelo, billionaires, afghanistan, beijing, biofuels, aliens
Character Analysis• Moretti: “protagonist is the character that minimized the
sum of the distances to all other vertices”• (But Moretti did it by hand!)
Character similarity analysis from A Dance with Dragons
Acknowledgements
Brent Kievet-Kylarword2word
Mike JonesBEAGLE
InPhO Team