Date post: | 16-Feb-2017 |
Category: |
Technology |
Upload: | mlconf |
View: | 1,020 times |
Download: | 1 times |
The Shape of DataAllison Gilmore
Principal Data Scientist
November 13, 2015
2Company Confidential & Proprietary
Data has shape.Shape has meaning.
You already know this.
Company Confidential & Proprietary
Shape as Organizing Principle
Company Confidential & Proprietary
Geometry or Topology?Geometry : Metric Topology : Locality
≅ ≅
Company Confidential & Proprietary
Topological Summaries Capture Shape
Lens
Company Confidential & Proprietary
Topological Summaries Capture Shape
Company Confidential & Proprietary
Topological Summaries Capture Shape
Company Confidential & Proprietary
Enhancing Traditional Methods
Company Confidential & Proprietary 9
Topological Summaries Capture Shape
Nodes are groups of similar data points.
Edges connect similar nodes.
Node position on the screen does not matter.
Company Confidential & Proprietary
Enhancing Traditional Methods
PCA sees 3 clusters.
Using PCA coordinates as lenses, we can see more.
Company Confidential & Proprietary
Topological Summary Shows 4 Clusters
Company Confidential & Proprietary
Disease State & Model Choice
David Schneider, Stanford Microbiology and Immunology
Company Confidential & Proprietary 14
Topological Model for Total Knee Replacement
Low length of stay
Low to moderate length of stay
Long length of stay
Company Confidential & Proprietary
Carepaths for Total Knee Replacement
16
Company Confidential & Proprietary 18
Beating* the Curse of Dimensionality
* I mean, there are always conditions.Niyogi, Smale, and Weinberger, A Topological View of Unsupervised Learning from Noisy Data,SIAM J. of Computing 20(2011) 646-663. http://math.uchicago.edu/~shmuel/noise.pdf
If a dataset is supported near a manifold, its key topological features can be detected from a sample whose size is independent of the dimension of ambient space.
Doesn’t matter!Dimension d < N
19Company Confidential & Proprietary
www.ayasdi.com
Company Confidential & Proprietary 20
Understanding Shape Improves Models
20
HighLow
Ground Truth Fraud Model Predicted Fraud
HighLow
Company Confidential & Proprietary 21
Topology Guides Model Creation