C O N F I D E N T I A L Apr 20, 2023 1

StarCite Engineering Weekly Meeting

StarCite Engineering

Feb 9, 2009

C O N F I D E N T I A L Apr 20, 2023 2

Agenda

• Quarterly Engineering top priorities

• Shanghai Engineering team weekly update – 5 minutes– Releases schedule overview

– Unit Test weekly progress chart

– Rally Iteration Weekly status

– Engineering Operation Metrics

– QA metrics

• Special Topics – 10 minutes – Complete Agile progress in SM – Leslie

– Code review process - Justin

• Q&A

C O N F I D E N T I A L Apr 20, 2023 3

Engineering Q1/09 Top 3 Priorities (aligned)

Priority Description Business Benefit Projects Involved

1

Performance, Stability and Quality

-Customer Satisfaction and Adoption

-Improved performance under increased load from grow growth and migration

-Improved quality with increased usage from growth and migration

-Reduce long term operation cost

-SM

-MarketView

-CBS – Search/UAS

-Mpoint.com

-Reporting

2 Migration

-Customer retention

-Reduced complexity

-Strategic advantage to single suite

-Spend Mgmt

-Attendee Mgmt

-MarketView

-Reporting

3 New Revenue

- By new features or new projects

-By enhanced features

-New product / services

-Attendee Mgmt (Event Marketing)

-Reporting (Buyer, Supplier)

-SM

-StarCite Express

-MKV

-More …

Absolutely has to happen – put best effort and team to meet the metrics and dates

C O N F I D E N T I A L Apr 20, 2023 4

Performance, Stability and Quality – Why, What and How

Why? • To improve internal quality to enhance both the functional and non-

functional quality of our products to satisfy customers• To gain business benefits

What to do?1. Effective code review including design review2. Unit test coverage and effectiveness 3. SQL profiling4. Code re-factoring

How?• To re-enforce code review, unit test and SQL profiling; to make them as

part of the development work• To provide guideline and good examples, to build good practice, • To be measurable; to know where we are today; to set near- and long-

term target; to track progress;

C O N F I D E N T I A L Apr 20, 2023 5

Release Schedule in Q1/2009

Project Release Date Version

SM Feb 18, 2009

March 29, 2009

12.15.002

12.16

Reporting Feb 20, 2009 1.7

G2 AM Feb 17, 2009

Feb 24, 2009

March 28, 2009

14.6.2

14.6.2 hotfix

14.6.4

MKV Feb 25, 2009

March 25, 2009

MKV86

MKV87

Mpoint.com March 9, 2009 (Target date) Mpoint.com 1.0.0.1

CBS – Search Feb 2, 2009 Search1.0.0.1

C O N F I D E N T I A L Apr 20, 2023 6

Unit Test Objectives

Priority Description Business Benefit

1 Quality improvementImproved quality level for all products. Reduction in regressions and customer issues

2Predictable, aggressive, re-factoring

Reduces risk and potential impact in aggressive performance and maintainability improvements

3Overhead reduction for development and testing

Shorter release cycles, more stories per sprint

4 Fail fast developmentFailures found sooner in development cycle. Allows faster course correction and more predictability in product delivery

C O N F I D E N T I A L Apr 20, 2023 7

Current Unit Test Coverage Numbers – by last Friday

Project Classes Line Coverage

Branch Coverage

Complexity

Mpoint.com 340 87% 90%

MarketView 613 49% 54% 1.709

Spend Management

2144 16% 11%

CBS – UAS* 0* 0* (92%) 0* (81%)

CBS - Search* 105 75% 65%

* Newly started project, has no unit test coverage yet

C O N F I D E N T I A L Apr 20, 2023 8

QA Automation – Accomplished in 2008

Product Category

Q4 2008

QTP QTP QTP %

SM

smoke 0 98 91.60%

SM UAT 78 413 64.10%

Maintenance 40 149 99.30%

         

MKV

smoke 0 267 100.00%

UAT - RFP WF 0 1583 79.70%

UAT - non RFP WF 252 755 56.40%

Maintenance 588 588 100.00%

         

G2AM

smoke 0 29 100.00%

UAT 46 46 43.40%

Configuration Test 0 9 45.00%

         

Reporting

Smoke 0 18 100.00%

UAT 73 245 29.80%

C O N F I D E N T I A L Apr 20, 2023 9

QA Automation – Q1 2009 Target

Product Category

Q1 2009

QTP

SM UAT 77%

MKV

UAT RFP WF 80%

UAT - non RFP WF 80%

G2 AM

UAT 100%

Configuration Test 100%

Reporting

smoke 100%

UAT 58%

100% Smoke test script get updated

C O N F I D E N T I A L Apr 20, 2023 10

Engineering Operation metrics

*3 customer discovered S0 – due to server out of memory.

Total Open/Closed Level1 (Last 7 days)

C O N F I D E N T I A L Apr 20, 2023 11

Engineering Operation metrics (cont.)

Level1 Status and Assignment (Last 7 days)

C O N F I D E N T I A L Apr 20, 2023 12

QA metrics – Q4 2008 release reviewDefect Resolution Efficiency - SM

0%

20%

40%

60%

80%

100%

SM release

Re

so

luti

on

%

Defect Resolution Efficiency - MKV

0%20%

40%60%

80%100%

MKV(8

31)

MKV(8

32)

MKV(8

33)

MKV(8

4)

MKV(8

401)

MKV(8

402)

MKV(8

41)

MKV(8

42)

MKV(8

421)

MKV(8

5)

MKV release

De

fec

t R

es

olu

tio

n %

C O N F I D E N T I A L Apr 20, 2023 13

QA metrics – Q4 2008 release review

Testing Efficiency - SM

0%20%40%60%80%

100%

MV (1

2.12

.1)

MV (1

2.13

)

MV (1

2.13

.1)

MV (1

2.13

.2)

MV (1

2.14

)

MV (1

2.14

.1)

MV (1

2.15

)

MV (1

2.15

.1)

SM release

% o

f d

efe

cts

in Q

A c

yc

le

ov

er

tota

l de

fec

ts

Testing Efficiency - MKV

0%20%40%60%80%

100%120%

MKV release

% o

f d

efe

cts

of

QA

cy

cle

o

ve

r to

tal d

efe

cts

C O N F I D E N T I A L Apr 20, 2023 14

QA metrics – Q4 2008 release review

Regression bug ratio - SM

0%

10%

20%

30%

40%

50%

60%

MV(12.12.1)

MV (12.13) MV(12.13.1)

SM release

% o

f re

gre

ss

ion

ov

er

tota

l d

efe

cts

Regression bug Ratio - MKV

0%10%20%30%40%50%60%70%

MKV(831)

MKV(832)

MKV(833)

MKV(84)

MKV(8401)

MKV release

% o

f re

gre

ss

ion

ov

er

tota

l d

efe

cts

C O N F I D E N T I A L Apr 20, 2023 15

QA metrics – Q4 2008 release review

Defects/Test case Ratio - MKV

0%2%4%6%8%

10%12%14%

MKV(8

31)

MKV(8

32)

MKV(8

33)

MKV(8

4)

MKV(8

401)

MKV(8

402)

MKV(8

41)

MKV(8

42)

MKV(8

421)

MKV(8

5)

MKV release

De

fec

t p

er

TC

C O N F I D E N T I A L Apr 20, 2023 16

Special Topics – Complete Agile

SM.12.15.2 – sprint 1

SM.12.15.2 – sprint 2

Big improvement is evident by comparing the iteration burn down charts

-- SM12.15.2 – sprint 2 shows healthier burn down chart than the SM12.15.2 – sprint 1

-- SM12.15.2 – sprint 2 accomplished much more story points by last day of

the iteration

C O N F I D E N T I A L Apr 20, 2023 17

Special Topics – Code Review processCross Code Review

Purpose: promote a culture “team excellent” of 2009 goals from Yang, and an attitude “I DO welcome anyone reviewing my codes and it can help me grow faster“.

Focus: most important technique hotspots:1. Out-of-memory issue identified by operation team2. Unexpected server restart identified by operation team3. Very long DB operations identified by SQL profiling log and app Log4. Legacy codes need re-factor and the new codes5. Other important issues identified by Team

Peer Code ReviewPurpose: improve code quality by regular code review activities  

Focus: 1. All new codes have to be reviewed and the activities must be tracked on

Rally. 2. Legacy codes are encouraged to have more code reviews and related

action plans

C O N F I D E N T I A L Apr 20, 2023 18

Special Topics – Code Review process

Encourage Plan

1. Performance bonus: code review efficiency (including frequency and quality) will be treated as one very important performance metric of team member.

2. Recognition: we will recognize the best engineers for the contribution on code review activities and the detailed plan will be decided later.

3. Leaning Credits: the engineers with good contribution on code review activities will get extra leaning credits to get encouraged.  

Introduce a web based tool for code review:

Review Board which developed by Vmvare(http://www.review-board.org/)

Evaluation is underway. If anyone is interested, please come forward and talk to Justin

C O N F I D E N T I A L Apr 20, 2023 19

Mr. Q&A