34
Patent Public Advisory Committee
Quarterly Meeting
Quality Initiative Update
May 5, 2016
Quality Agenda
• Patent Quality Community Symposium
• Quality Metrics
• Re-evaluate AFCP 2.0 and Pre-appeal
• Topic Submission for Case Studies
• External Survey Results
3
Patent Quality Community Symposium: Empowering Innovation Through Enhanced Quality
Richard Seidel
Acting Deputy Commissioner for Patent Administration
Patent Quality Community
Symposium
• Wednesday, April 27, 2016
• Where
– USPTO Alexandria
– All four regional offices
– Webcast
• Participation: Over 2,2005
Patent Quality Community
Symposium
Featured Presentations
• Updates on Enhanced Patent Quality
Initiative (EPQI)
• USPTO’s efforts to use Big Data
• Quality Metrics for FY2016
• Master Review Form Workshop (MRF)6
Quality Metrics
Richard Seidel
Acting Deputy Commissioner for Patent Administration
Quality Metrics Redefined
Final Disposition Compliance
In-Process Compliance
First Action (FAOM) Review
Search Review
Quality Index Reporting (QIR)
External Quality Survey
Internal Quality Survey
Composite Score
FY 2011 - FY 2015Product Indicators
Master Review FormCapturing both correctness and clarity of examiners’ final work
product using uniform criteria gathered in a single database
Process Indicators
Transactional QIRTracking the efficiency and consistency of our processes
(for example, to identify “churning”)
Perception Indicators
Survey ResultsContinuing to internally and externally poll perceptions of
patent quality
FY 2016
8
Quality Metric Data Source:
Product Indicators
Correctness
Clarity
FY 2016 Key Product Metrics FY 2016
9
Product Indicators
Master Review FormCapturing both correctness and clarity of examiners’ final work
product using uniform criteria gathered in a single database
Process Indicators
Transactional QIRTracking the efficiency and consistency of our processes
(for example, to identify “churning”)
Perception Indicators
Survey ResultsContinuing to internally and externally poll
perceptions of patent quality
Quality Metric Data Source:
Process Indicators
FY 2016 Key Process Indicators
FY 2016
10
Reopening Prevention
Consistency of
Decision-Making
Rework Reduction
Perception Indicators
Survey ResultsContinuing to internally and externally poll
perceptions of patent quality
Product Indicators
Master Review FormCapturing both correctness and clarity of examiners’ final work
product using uniform criteria gathered in a single database
Process Indicators
Transactional QIRTracking the efficiency and consistency of our processes
(for example, to identify “churning”)
Quality Metric Data Source:
Perception Survey Results
FY 2016 Vital Perception Indicators
FY 2016
11
Root Cause Analysis
Validation/Verification
Product Indicators
Master Review FormCapturing both correctness and clarity of examiners’ final work
product using uniform criteria gathered in a single database
Process Indicators
Transactional QIRTracking the efficiency and consistency of our processes
(for example, to identify “churning”)
Perception Indicators
Survey ResultsContinuing to internally and externally poll
perceptions of patent quality
Quality Metrics
12
• Federal Register Notice published on March 25, with comments due May 24– Requesting feedback on:
• Decision to replace Composite Quality Score with individual metrics
• How to objectively measure patent examination quality
• Standardized Master Review Form
Quality Metrics website: http://www.uspto.gov/patent/initiatives/quality-metrics
Contact Us: [email protected]
Questions & Comments
Richard Seidel
Acting Deputy Commissioner for Patent Administration
(571) 272-2950
Re-evaluate AFCP 2.0 and Pre-appeal
Remy Yucel
Assistant Deputy Commissioner for Patent Operations
Re-evaluate AFCP 2.0 and Pre-appeal
• Tests how some of the best attributes of
the AFCP 2.0 and the Pre-Appeal pilots can
be combined to give both applicants and
examiners additional information
• Increased understanding of the issues will
lead to more accurate decisions on
subsequent courses of action15
Re-evaluate AFCP 2.0 and Pre-appeal
• Features under consideration:
– Available within 2 months of final rejection
– Panel, including a neutral party
– Applicant participation to present arguments as in Pre-Appeal (5-page document) or claim amendments
– More information on panel decision (i.e. grounds of rejection withdrawn or maintained, claims rejected, allowed, additional brief comments)
16
Questions & Comments
Remy Yucel
Assistant Deputy Commissioner for Patent Operations
(571) 272-0700
Topic Submission for Case Studies
Anthony Caputa
Director, Office of Patent Quality Assurance (OPQA)
Topic Submission for Case Studies:
Pilot Summary
• Federal Register Notice initiated the program on December 21, 2015
• Submissions were accepted through February 12, 2016
• USPTO invited stakeholders to submit patent quality-related topics for study
19
What is a Case Study?
• Review of a single, quality-related issue
• Tailored to the selected issue
• Performed by USPTO
– Distinct from standard reviews completed by
the Office of Patent Quality Assurance (OPQA)
20
Topic Submission for Case Studies:
Program Goals
• Use stakeholder experience to provide USPTO with a
wider range of topics to consider for a case study
• Use study results to better understand and enhance
quality of USPTO work products and processes to:
– Identify quality issues and examples of examination
best practices
– Reveal areas where further training may be needed
21
Topics Selected for Case Study
1. Compliance of rejections with 35 U.S.C. 101 Official Guidance
2. Consistency of application of 35 U.S.C. 101 across Art Units/Technology Centers
3. Use of compact prosecution when making 35 U.S.C. 101 rejections
4. Correctness and clarity of motivation statements in 35 U.S.C. 103 rejections
5. Enforcement of 35 U.S.C. 112(a) written description in continuing applications
6. Consistent treatment of claims after May 2014 35 U.S.C. 112(f) training
22
Topic Submission - Resources
• Topic Submission for Case Studies Webpage: http://www.uspto.gov/patent/initiatives/topic-submission-case-studies-pilot-program
• Enhanced Patent Quality Initiative Webpage: http://www.uspto.gov/patent/initiatives/enhanced-patent-quality-initiative
• Contact us at [email protected]
23
Questions & Comments
Anthony Caputa
Director, Office of Patent Quality Assurance (OPQA)
(571) 272-0829
External Quality Survey (EQS)
Martin Rater
Chief Statistician, Office of Patent Quality Assurance (OPQA)
External Quality Survey (EQS)
• Conducted semi-annually since 2006– Most recent survey Q1 in FY16
• Surveys 3,000 frequent-filing customers with each survey
• Has been included in Patent Quality Composite (FY2011-15)
• Continues to be a vital quality indicator as we transition to new quality metrics in FY17
26
Perception of Product:
Quality of Rejections Made
27
0%
10%
20%
30%
40%
50%
60%
70%
FY07Q1
FY07Q3
FY08Q3
FY09Q1
FY09Q3
FY10Q1
FY10Q3
FY11Q1
FY11Q3
FY12Q1
FY12Q3
FY13Q1
FY13Q3
FY14Q1
FY14Q3
FY15Q1
FY15Q3
FY16Q1
6e. 35 U.S.C. 112Rejections, Paragraph 2
6b. 35 U.S.C. 102Rejections
6d. 35 U.S.C. 112Rejections, Paragraph 1
6a. 35 U.S.C. 101Rejections
6c. 35 U.S.C. 103Rejections
Q6: Frequency of Technically, Legally, and Logically Sound Rejections
(Percent reporting “Most of the time” or “All of the time” of the time)
335 U.S.C. 112(b)
5 U.S.C. 102
35 U.S.C. 112(a)
35 U.S.C. 101
35 U.S.C. 103
Rejections under
Perception of Product and Process:
Advancing Prosecution
28
Q10: In your experience, what have examiners done that has helped advance prosecution?
192
151
146
117
108
528
0 100 200 300 400 500 600
Interviews at all stages
Initiate contact with Applicant
Willing to discuss via phone/email
Well prepared for meetings
Well written responses
Collaborative, constructive, makes suggestions
Perception of Product:
Consistency
29
Q9: In the past 3 months, have you experienced problems with the consistency of
examination quality from one examiner to another?
Large degree of
inconsistency
Small degree of
inconsistency
No inconsistency
0%
10%
20%
30%
40%
50%
60%
FY07
Q1
FY07
Q3
FY08
Q3
FY09
Q1
FY09
Q3
FY10
Q1
FY10
Q3
FY11
Q1
FY11
Q3
FY12
Q1
FY12
Q3
FY13
Q1
FY13
Q3
FY14
Q1
FY14
Q3
FY15
Q1
FY15
Q3
FY16
Q1
Perception of Overall Quality
30
31%33%
30%27% 28%
32%
40%38%
42%45%
47%
51% 52% 51% 51% 50%47%
54%
21% 20%23%
25% 24%
18%
11%14% 14%
9% 9% 8% 9% 9% 8% 9%11%
9%
0%
10%
20%
30%
40%
50%
60%
FY07
Q1
FY07
Q3
FY08
Q3
FY09
Q1
FY09
Q3
FY10
Q1
FY10
Q3
FY11
Q1
FY11
Q3
FY12
Q1
FY12
Q3
FY13
Q1
FY13
Q3
FY14
Q1
FY14
Q3
FY15
Q1
FY15
Q3
FY16
Q1
Percent reporting
"good" or "excellent"
Percent reporting
"poor" or "very poor"
Q7: Percent Positive and Negative Ratings of Overall Examination Quality in Past 3 Months
Link between Perception of
Consistency and Overall Quality
31
75%65%
27%
23%32%
53%
3% 3%
20%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
No Inconsistency Small Degee of
Inconsistency
Large Degree of
Inconsistency
Poor or Very Poor
Fair
Good or Excellent
Overall Quality
Perception
Using EQS for Validating the
Quality Metric
32
• Utilize the External Quality Survey as a snapshot of
stakeholders’ perceptions
• Assure alignment of the quality data underlying our
metrics and our external stakeholders’ perceptions
• Exploit the flexibility of the Master Review Form to
capture data points that reflect patent quality
Questions & Comments
Martin Rater
Chief Statistician, Office of Patent Quality Assurance (OPQA)[email protected]
(571) 272-5966
