C O N F I D E N T I A L Apr 20, 2023 1
StarCite Engineering Weekly Meeting
StarCite Engineering
Feb 9, 2009
C O N F I D E N T I A L Apr 20, 2023 2
Agenda
• Quarterly Engineering top priorities
• Shanghai Engineering team weekly update – 5 minutes– Releases schedule overview
– Unit Test weekly progress chart
– Rally Iteration Weekly status
– Engineering Operation Metrics
– QA metrics
• Special Topics – 10 minutes – Complete Agile progress in SM – Leslie
– Code review process - Justin
• Q&A
C O N F I D E N T I A L Apr 20, 2023 3
Engineering Q1/09 Top 3 Priorities (aligned)
Priority Description Business Benefit Projects Involved
1
Performance, Stability and Quality
-Customer Satisfaction and Adoption
-Improved performance under increased load from grow growth and migration
-Improved quality with increased usage from growth and migration
-Reduce long term operation cost
-SM
-MarketView
-CBS – Search/UAS
-Mpoint.com
-Reporting
2 Migration
-Customer retention
-Reduced complexity
-Strategic advantage to single suite
-Spend Mgmt
-Attendee Mgmt
-MarketView
-Reporting
3 New Revenue
- By new features or new projects
-By enhanced features
-New product / services
-Attendee Mgmt (Event Marketing)
-Reporting (Buyer, Supplier)
-SM
-StarCite Express
-MKV
-More …
Absolutely has to happen – put best effort and team to meet the metrics and dates
C O N F I D E N T I A L Apr 20, 2023 4
Performance, Stability and Quality – Why, What and How
Why? • To improve internal quality to enhance both the functional and non-
functional quality of our products to satisfy customers• To gain business benefits
What to do?1. Effective code review including design review2. Unit test coverage and effectiveness 3. SQL profiling4. Code re-factoring
How?• To re-enforce code review, unit test and SQL profiling; to make them as
part of the development work• To provide guideline and good examples, to build good practice, • To be measurable; to know where we are today; to set near- and long-
term target; to track progress;
C O N F I D E N T I A L Apr 20, 2023 5
Release Schedule in Q1/2009
Project Release Date Version
SM Feb 18, 2009
March 29, 2009
12.15.002
12.16
Reporting Feb 20, 2009 1.7
G2 AM Feb 17, 2009
Feb 24, 2009
March 28, 2009
14.6.2
14.6.2 hotfix
14.6.4
MKV Feb 25, 2009
March 25, 2009
MKV86
MKV87
Mpoint.com March 9, 2009 (Target date) Mpoint.com 1.0.0.1
CBS – Search Feb 2, 2009 Search1.0.0.1
C O N F I D E N T I A L Apr 20, 2023 6
Unit Test Objectives
Priority Description Business Benefit
1 Quality improvementImproved quality level for all products. Reduction in regressions and customer issues
2Predictable, aggressive, re-factoring
Reduces risk and potential impact in aggressive performance and maintainability improvements
3Overhead reduction for development and testing
Shorter release cycles, more stories per sprint
4 Fail fast developmentFailures found sooner in development cycle. Allows faster course correction and more predictability in product delivery
C O N F I D E N T I A L Apr 20, 2023 7
Current Unit Test Coverage Numbers – by last Friday
Project Classes Line Coverage
Branch Coverage
Complexity
Mpoint.com 340 87% 90%
MarketView 613 49% 54% 1.709
Spend Management
2144 16% 11%
CBS – UAS* 0* 0* (92%) 0* (81%)
CBS - Search* 105 75% 65%
* Newly started project, has no unit test coverage yet
C O N F I D E N T I A L Apr 20, 2023 8
QA Automation – Accomplished in 2008
Product Category
Q4 2008
QTP QTP QTP %
SM
smoke 0 98 91.60%
SM UAT 78 413 64.10%
Maintenance 40 149 99.30%
MKV
smoke 0 267 100.00%
UAT - RFP WF 0 1583 79.70%
UAT - non RFP WF 252 755 56.40%
Maintenance 588 588 100.00%
G2AM
smoke 0 29 100.00%
UAT 46 46 43.40%
Configuration Test 0 9 45.00%
Reporting
Smoke 0 18 100.00%
UAT 73 245 29.80%
C O N F I D E N T I A L Apr 20, 2023 9
QA Automation – Q1 2009 Target
Product Category
Q1 2009
QTP
SM UAT 77%
MKV
UAT RFP WF 80%
UAT - non RFP WF 80%
G2 AM
UAT 100%
Configuration Test 100%
Reporting
smoke 100%
UAT 58%
100% Smoke test script get updated
C O N F I D E N T I A L Apr 20, 2023 10
Engineering Operation metrics
*3 customer discovered S0 – due to server out of memory.
Total Open/Closed Level1 (Last 7 days)
C O N F I D E N T I A L Apr 20, 2023 11
Engineering Operation metrics (cont.)
Level1 Status and Assignment (Last 7 days)
C O N F I D E N T I A L Apr 20, 2023 12
QA metrics – Q4 2008 release reviewDefect Resolution Efficiency - SM
0%
20%
40%
60%
80%
100%
SM release
Re
so
luti
on
%
Defect Resolution Efficiency - MKV
0%20%
40%60%
80%100%
MKV(8
31)
MKV(8
32)
MKV(8
33)
MKV(8
4)
MKV(8
401)
MKV(8
402)
MKV(8
41)
MKV(8
42)
MKV(8
421)
MKV(8
5)
MKV release
De
fec
t R
es
olu
tio
n %
C O N F I D E N T I A L Apr 20, 2023 13
QA metrics – Q4 2008 release review
Testing Efficiency - SM
0%20%40%60%80%
100%
MV (1
2.12
.1)
MV (1
2.13
)
MV (1
2.13
.1)
MV (1
2.13
.2)
MV (1
2.14
)
MV (1
2.14
.1)
MV (1
2.15
)
MV (1
2.15
.1)
SM release
% o
f d
efe
cts
in Q
A c
yc
le
ov
er
tota
l de
fec
ts
Testing Efficiency - MKV
0%20%40%60%80%
100%120%
MKV release
% o
f d
efe
cts
of
QA
cy
cle
o
ve
r to
tal d
efe
cts
C O N F I D E N T I A L Apr 20, 2023 14
QA metrics – Q4 2008 release review
Regression bug ratio - SM
0%
10%
20%
30%
40%
50%
60%
MV(12.12.1)
MV (12.13) MV(12.13.1)
SM release
% o
f re
gre
ss
ion
ov
er
tota
l d
efe
cts
Regression bug Ratio - MKV
0%10%20%30%40%50%60%70%
MKV(831)
MKV(832)
MKV(833)
MKV(84)
MKV(8401)
MKV release
% o
f re
gre
ss
ion
ov
er
tota
l d
efe
cts
C O N F I D E N T I A L Apr 20, 2023 15
QA metrics – Q4 2008 release review
Defects/Test case Ratio - MKV
0%2%4%6%8%
10%12%14%
MKV(8
31)
MKV(8
32)
MKV(8
33)
MKV(8
4)
MKV(8
401)
MKV(8
402)
MKV(8
41)
MKV(8
42)
MKV(8
421)
MKV(8
5)
MKV release
De
fec
t p
er
TC
C O N F I D E N T I A L Apr 20, 2023 16
Special Topics – Complete Agile
SM.12.15.2 – sprint 1
SM.12.15.2 – sprint 2
Big improvement is evident by comparing the iteration burn down charts
-- SM12.15.2 – sprint 2 shows healthier burn down chart than the SM12.15.2 – sprint 1
-- SM12.15.2 – sprint 2 accomplished much more story points by last day of
the iteration
C O N F I D E N T I A L Apr 20, 2023 17
Special Topics – Code Review processCross Code Review
Purpose: promote a culture “team excellent” of 2009 goals from Yang, and an attitude “I DO welcome anyone reviewing my codes and it can help me grow faster“.
Focus: most important technique hotspots:1. Out-of-memory issue identified by operation team2. Unexpected server restart identified by operation team3. Very long DB operations identified by SQL profiling log and app Log4. Legacy codes need re-factor and the new codes5. Other important issues identified by Team
Peer Code ReviewPurpose: improve code quality by regular code review activities
Focus: 1. All new codes have to be reviewed and the activities must be tracked on
Rally. 2. Legacy codes are encouraged to have more code reviews and related
action plans
C O N F I D E N T I A L Apr 20, 2023 18
Special Topics – Code Review process
Encourage Plan
1. Performance bonus: code review efficiency (including frequency and quality) will be treated as one very important performance metric of team member.
2. Recognition: we will recognize the best engineers for the contribution on code review activities and the detailed plan will be decided later.
3. Leaning Credits: the engineers with good contribution on code review activities will get extra leaning credits to get encouraged.
Introduce a web based tool for code review:
Review Board which developed by Vmvare(http://www.review-board.org/)
Evaluation is underway. If anyone is interested, please come forward and talk to Justin
C O N F I D E N T I A L Apr 20, 2023 19
Mr. Q&A