Mastergoal Machine Learning Environment

Phase 1 Completion AssessmentMSE Project

Kansas State University

Alejandro Alliana

Deliverables

• Vision Document v 0.1.0

• Project Plan 0.1.0

• Software Quality Assurance Plan 0.1.0

• Prototype

Mastergoal

• Board game with discrete states.

• Played at different levels.

• High branching factor.

• New in AI research.

Project Goals

• Provide an environment to create, repeat, save experiments for creating strategies for playing Mastergoal using ML techniques.

• Try different AI techniques in the environment of the game

Background

• Traditional approaches– Search in the state space S applying actions A(st) to

the states and Evaluating the generated states st+1

using a hand crafted evaluation function

• Reinforcement Learning– Unsupervised learning.– Temporal difference learning.– Successful with Backgammon.– Problems with some games such as Chess and Go.– TD-Leaf, TD(μ)

Risks

• Inexperience with some algorithms and programming language

• Exploration vs. exploitation

• Computational Cost of Evaluation Functions

• Quality

Prototypes demonstration

Constraints

• Export strategies to be used in the Mastergoal plugin environment.

• CPP programming language

Requirements

• Experiment Management

• Training strategy

• Export Strategy

• Explore game

System Components

Experiment Management

Other Use Cases

Documentation standards

• UML Diagrams

• Scenario description

• Coding Standards following the C++ standards

• Commentary standards following Code Conventions for the Java Programming Language.

Testing Standards

• Unit testing– CppTest

• Component testing

• Integration Testing

• Performance Testing

• Testing plan

Version Control

• SVN Repository

• Maven directory Structure standard

• Tortoise SVN Client

Tools

• IDE– Microsoft Visual Studio

• Modeling– Rational Rose– Gliffy.com

• Documentation– Microsoft Word

• Code control– Tortoise SVN

• Managing– Process Dashboard

– Microsoft Project

Cost Estimate

• COCOMO

• COCOMO II

• Use case points

COCOMO

• Effort = 3.2 EAF (Size) 1.05

• Time = 2.5 (Effort) 0.38

• Where:– Effort is the number of staff months– EAF is the product of 15 effort adjustment

factors.– Size is the number of delivered source

instructions in KLOC.

Cocomo – Effort Adjustment factors

Id Effort Adjustment Factor Parameter Range

Potential Impact Value Selected Reasoning

RELY Required reliability 0.75 - 1.40 1.87 1.00 Nominal - The application is reliability is not critical

DATA Database size 0.94 - 1.16 1.23 1.00 Nominal -Database access to store games

CPLX Product complexity 0.70 – 1.65 2.36 1.15 High – Product contains reinforcement learning algorithms.

TIME Execution time constraint 1.00 – 1.66 1.66 1.11 High – Experiments must not take too long, since the application is already computationally intensive.

STOR Main Storage Constraint 1.00 – 1.56 1.56 1.00 Nominal

VIRT Virtual machine volatility 0.87 – 1.30 1.49 1.00 Nominal

TURN Computer turnaround time 0.87 – 1.15 1.32 1.00 Nominal

ACAP Analyst capability 1.46 – 0.71 2.06 0.86 High. Developer has adequate experience.

AEXP Applications experience 1.29 – 0.82 1.57 1.10 Low. Some of the components are new to the developer

PCAP Programmer capability 1.42 – 0.70 2.03 1.00 Nominal

VEXP Virtual machine experience 1.21 – 0.90 1.34 1.00 Nominal. Developer has adequate experience with OS systems and tools.

LEXP Language experience 1.14 – 0.95 1.20 1.07 Low. Developer is new to the C++ language.

MODP Use of modern practices 1.24 – 0.82 1.51 0.91 High. The process will follow modern practices.

TOOL Use of software tools 1.24 – 0.83 1.49 0.91 High.

SCED Required development schedule 1.23 – 1.10 1.23 1.10 Low. Project is on a constrained schedule

COCOMO Estimate

• Estimated KLOC (7.5)

• Effort = 3.2 (1.18) (7.5) 1.05

• Effort= 31.32 staff months

• Time = 2.5 (Effort) 0.38

• Time = 9.25 months

COCOMO II

• COCOMO II defines three models for cost estimation:– Applications composition model– Early design model– Post-Architecture model.

Application Composition Model

• Assess Object-Counts

• Classify each object instance into simple, medium and difficult and weight them.

• Determine Object-Points

• Estimate percentage of reuse

• Determine a productivity rate

• Compute the estimated person-months

Application Composition Model

• PM = 39/7 = 5.57 Person months – (2.25 ~ 11.07 months)

Early Design Model

• Effort = 2.45 EArch (Size)P

• Where:– Effort = number of staff-moths– EArch = is the product of seven early design

effort adjustment factors– Size = number of function points or KLOC– P are the scaling factors.

Post Architecture model

• Effort = 2.45 (Eapp) (Size)P

– Effort = number of staff-moths– EArch = is the product of seventeen post

architecture effort adjustment factors – Size = number of function points or KLOC– P = process exponent, same as the early

design model.

• Effort = 33.99 staff months• Time = 9.54 months (7.632 ~ 11.93)

Project Schedule

Phase Two Deliverables

• Vision document

• Project Plan

• Test Plan

• Architecture Design

• Formal Requirements Specification

• Formal Technical Inspection

• Executable Architecture Prototype.

• Questions

End of presentation

Application Composition Model

Object Classification Object Points

Main screen Medium 2

Game exploration screen Medium 2

Export strategy screen Simple 1

Experiments status report Simple 2

Explore game screen Medium 2

Game component Complex 10

Search component Complex 10

Learning component Complex 10

TOTAL 39

Scaling factors

Scale Factor Abbreviation Value.

Precedentedness PREC Nominal – 3

Development Flexibility FLEX Low – 4

Architecture risk resolution RESL Nominal – 3

Team cohesion TEAM Extra high – 0

Frameworks Studied

• Knight Cap

• Neuro Draugths

• RL Glue