Distributed Monitoring & MiningOMSE PRACTICUM II FINAL PRESENTATIONJUNE 6, 2013
THOMAS MOONEY SHAILESH SHIMPI AHMED OSMAN ISAAC PENDERGASSREQUIREMENTS ENGINEER QA MANAGER/ TEST
ENGINEER ARCHITECT
PROJECT MANAGER
“”
… as software systems become more complex, automated analysis may be the only way that we can keep up with their size and scope.
Our Purpose
Project Status Architecture Google Prediction Engine Application Demonstration Project Review What’s next?
To augment current collaboration tools by collecting and modeling historical project data to alert project stakeholders when signs of trouble are detected.
Project Status Deliverables
Planned Project Proposal Software Project Management Plan (SPMP) Concept of Operations (ConOps) Software Requirements Specification (SRS) Software Architecture Document (SAD) Software Quality Assurance Plan (SQAP) User Manual Metrics Document Prediction Application
Project Status (cont.) Deliverables
Completed Project Proposal Software Project Management Plan (SPMP) Concept of Operations (ConOps) Software Requirements Specification (SRS) Software Architecture Document (SAD) Software Quality Assurance Plan (SQAP) User Manual Metrics Document Prediction Application
Architecture
Architectural Highlights – Goals and Constraints Useful to future architects and designers Written with MS .NET, but deployed with open source technologies
(Mono, MySQL). Interfaces with 3rd party APIs (Google, Assembla). Minimize the impact of anticipated changes
Changes to the Assembla API Changes to the Google API Changes to the metrics used to construct the predictive model Changes to the content of the prediction report.
Architecture – Components
Google Prediction Engine What is Google Predictive API ?
The Prediction API provides pattern-matching and machine learning capabilities. Given a set of data examples to train against.
It is using supervised learning It uses classification and regression pattern recognition https://developers.google.com/prediction/docs/getting-started
Why do we use it? We need to classify projects either in risk or in the right path Free to use (for 6 months) and has many API flavors (.NET, python …)
Google Prediction EngineHow is it working?
Create Training Data (Most Important)
Upload the training data to Google Cloud
Train the model from the data file
Start to get the prediction by API
Status Milestones Completed
Average Closed Tickets
Average Open Tickets
AROT ATP OTV MTBT MT
GOOD 2 2 2 0 3 0 0.000189 0GOOD 1 0 0 0 0 0 0 0
WARNING 8 8.857143 9 0.142857 2.955882 1.573232 0.645121 0
WARNING 6 0.75 1 0.25 3 0 0.001831 0
Demonstration
Project Review Model Fidelity Probably the most important aspect How do we define success? Is it constant?
DMMs Criterion: 75% of completed milestones, completed on-time.
Project Review Metrics Selection The “right” metrics are not always available. Extra care and, in some cases, additional infrastructure is required
in the gathering of the metrics and their handling during analysis. Extending the system to support more repositories increases this
challenge due to differing formats. Repository usage does not always reflect serious development
effort.
We must choose metrics based on data that has a reasonable probability of being recorded in order to build models that are useful in determining user-project status.
Project Review A Look at the Metrics Total Number of Milestones Opened Tickets per Milestone Closed Tickets per Milestone Abandoned Tickets per Milestone Average Ticket Priority Opened Tickets per Day Mean Time Between Tickets Mean Time Between High Priority Tickets
Project Review (cont.) Organization Roles assigned on volunteer basis Vote would be cast in the event two members wanted same role Every voice held the same weight
Design and Development Infrastructure Documents housed on Skydrive Wiki Contains references to all documents and project information Mirrored code repository Continuous Integration Server
Project Review (cont.) Communication Decided to perform duties in distributed manner
The DMM Team has never met face to face
Email supplanted forums to become main medium of communication
Relatively few misunderstandings
Always held meetings for issues that required "complex" communications
What's Next?
Increase metric relevancy and quantity
Incorporate auto-notification scheme
Provide more reporting options
Provide mechanism for user defined analysis
Questions?
DMM Home Page
Assembla Login Screen
Select Project Space
Analysis Report
PDF Report