Building the Autodesk Design Graph using Apache SparkYotto Koga

3

4

• focus on the solution • catch problems early • gain insights

I can…

5

martingrandjean

6

Project Communicationsmarkups, workflows, activity,

schedules, BOM, etc.

Company Datalibraries, AVLs, support data

People

Design FilesBREPs, constraints, metadata,

revisions, relationships

Parts

Where to start?

7

8

9

10

Parts ~ Words

11

12

Fasteners

13

FastenersPower Transmission

14

gear

shaft

pin

cam

shaft

arm

pinpiston

15

Initial Design Graph

16

How do we identify parts?

17

UNRELIABLE

18

19

20

Shape is our most reliable and data rich piece of information

21

22

23

24

R1024

M. Kazhdan et al,”Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors"

Spherical Harmonic Shape Descriptor

25

Bigram Shape Descriptor

R4110

26

27

28

29

30

31

32

Data Source Data Transformation and Machine Learning Pipeline

Design Graph

33

Design Graph

Query Optimized Graph Database

Front End API Server

Locality Sensitive Hash

Custom Redis

Elastic Search

Search Engine Index

34

Data Source Data Transformation and Machine Learning Pipeline

Design Graph

35

Batch Data Source

Design Graph

Part Extraction

Sp Harmonics Descriptor Generation

Data Cleanup

Bigram Description Generation

Part Clustering/Labeling

Design B of P Model Generation

Design Clustering/Labeling

Design Graph

Generation

36

≈ Mesh Processing C++

≈

37

Mesh Processing C++

≈

38

Mesh Processing C++

RDD Transform (Batch)

Input RDD

≈

39

Mesh Processing C++

RDD Transform (Batch)

Output RDD

≈

40

Mesh Processing C++

RDD Transform (Batch)

Input RDD

≈

41

Mesh Processing C++

RDD Transform (Batch)

Output RDD

≈

42

Mesh Processing C++

RDD Transform (Batch)

Input RDD

≈

43

Mesh Processing C++

RDD Transform (Batch)

Output RDD

≈

44

Mesh Processing C++

RDD Transform (Batch)

Final RDD

Fail

45

Bonus: Isolated Tests

46

Part Extraction

Sp Harmonics Descriptor Generation

Data Cleanup

Bigram Description Generation

Part Clustering/Labeling

Design B of P Model Generation

Design Clustering/Labeling

Design Graph

Generation

47

Part Extraction

Sp Harmonics Descriptor Generation

Data Cleanup

Bigram Description Generation

Part Clustering/Labeling

Design B of P Model Generation

Design Clustering/Labeling

Design Graph

Generation

Design Graph Module

Persistent Lineage (S3)

48

Part Extraction

Sp Harmonics Descriptor Generation

Data Cleanup

Bigram Description Generation

Part Clustering/Labeling

Design B of P Model Generation

Design Clustering/Labeling

Design Graph

Generation

Persistent Lineage (S3)

Bigram Different

Parameters

Part Clustering/Labeling

Design B of P Model Generation

Design Clustering/Labeling

Design Graph

Generation

New Mesh Cleanup

Bigram Description Generation

Part Clustering/Labeling

Design B of P Model Generation

Design Clustering/Labeling

Design Graph

Generation

49

Demo

50

Part Extraction

Sp Harmonics Descriptor Generation

Data Cleanup

Bigram Description Generation

Design B of P Model Generation

Design Clustering/Labeling

Design Graph

Generation

Persistent Lineage (S3)

Design Graph Module

Batch DataPart

Clustering/Labeling

Classifier

Incremental Data Stream

51

Part Extraction

Sp Harmonics Descriptor Generation

Data Cleanup

Bigram Description Generation

Design B of P Model Generation

Design Clustering/Labeling

Design Graph

Generation

Persistent Lineage (S3)

Design Graph Module

Data Stream Part Clustering/Labeling

Queue

52

The Team!

Mike Haley

Senthil Gandhi

Tara Adiseshan

Federico Vezzeso

Morgan Dornbush

Carlos Olguin

Santiago Sanchez Khoa Tran

Yotto Koga

53