Go with the flow: optimizing voice recognition to streamline

workflow

Go with the flow: optimizing voice recognition to streamline

workflow

Jeffrey Chenoweth MDSaint Louis VAMC

Kim Wilson MDTucson VAMC

Jeffrey Chenoweth MDSaint Louis VAMC

Kim Wilson MDTucson VAMC

Voice recognition to streamline workflow

• Jeffrey Chenoweth MD– Saint Louis VAMC

• Kim Wilson MD– Tucson VAMC

• Jeffrey Chenoweth MD– Saint Louis VAMC

• Kim Wilson MD– Tucson VAMC

Voice recognition to streamline workflow

• Driving forces behind VR

• Pushback – VR controversy

• VR development• Case study – VR

implementation• PACS setup and

reporting with VR– Kim Wilson MD

• Practical points for improving Radiologist workflow

• Future development of VR

• Driving forces behind VR

• Pushback – VR controversy

• VR development• Case study – VR

implementation• PACS setup and

reporting with VR– Kim Wilson MD

• Practical points for improving Radiologist workflow

• Future development of VR

Radiology reporting

• Basics unchanged in last century

• Product not images but report

• Communication to improve patient care

• Penultimate step in Radiology process

• Final step -- clinician action

• Basics unchanged in last century

• Product not images but report

• Communication to improve patient care

• Penultimate step in Radiology process

• Final step -- clinician action

If goals of reporting are unchanged why do we need VR

now?

Why VR?

• Absence of skilled transcriptionists?

• Transcription cost?– Probably not

• Absence of skilled transcriptionists?

• Transcription cost?– Probably not

Why VR?

• Improved report turnaround time

• Fewer report errors

• Improved report turnaround time

• Fewer report errors

Why VR?

• Transcription turnaround time

– Cassette tapes: week – 10 days

– Digital dictation: hours – 3 days

– VR: minutes• Decreases calls for

preliminary read• Clinicians expect

immediate report availability

• Transcription turnaround time

– Cassette tapes: week – 10 days

– Digital dictation: hours – 3 days

– VR: minutes• Decreases calls for

preliminary read• Clinicians expect

immediate report availability

Why VR?

• Improved report turnaround time

• Improved patient care

• Makes the Radiology report relevant

• Improved report turnaround time

• Improved patient care

• Makes the Radiology report relevant

Why VR?

• Problems with the traditional report correction editing process– Outside normal Radiologist workflow– Disruptive– Time consuming

• Problems with the traditional report correction editing process– Outside normal Radiologist workflow– Disruptive– Time consuming

Why VR?

• Error prone traditional report correction – editing process

– Time lag forgetfulness• Grammar checking vs.

content errors– Right – left errors– Date errors– DHCP blue screen daze

• After 50 + reports, how closely are you reading the report?

• Error prone traditional report correction – editing process

– Time lag forgetfulness• Grammar checking vs.

content errors– Right – left errors– Date errors– DHCP blue screen daze

• After 50 + reports, how closely are you reading the report?

Why VR?

• Report completion while image is in front of Radiologist

• Immediate error correction

• Once you’re done, you’re done

• Immediate report availability

• Report completion while image is in front of Radiologist

• Immediate error correction

• Once you’re done, you’re done

• Immediate report availability

VR controversy

VR controversy – Radiologist’s view

• Increased dictation time

• Increased error rate vs. good transcriptionist

• Removes focus on images

• Increased dictation time

• Increased error rate vs. good transcriptionist

• Removes focus on images

VR controversy – accuracy rate

• Is 95% acceptable?

• 90% of all reports have errors prior to sign off

• 10 % of reports have errors with transcriptionists

– J Digit Imaging Jun 2007

• Is 95% acceptable?

• 90% of all reports have errors prior to sign off

• 10 % of reports have errors with transcriptionists

– J Digit Imaging Jun 2007

VR economically justified?

• Decreased Radiologist productivity

– 50% longer dictation time– 24% shorter reports

– J Digit Imaging Jun 2007

• Decreased Radiologist productivity

– 50% longer dictation time– 24% shorter reports

– J Digit Imaging Jun 2007

VR – economically justified?

• Replacing lower paid transcriptionists with highly paid physicians

– Greater Radiologist productivity transcriptionists more cost effective than VR

– “… which course of action makes the most economic sense… is not always obvious.”

– JACR 2007; 4: 890

• Replacing lower paid transcriptionists with highly paid physicians

– Greater Radiologist productivity transcriptionists more cost effective than VR

– “… which course of action makes the most economic sense… is not always obvious.”

– JACR 2007; 4: 890

VR – two decades of controversy

• “Has considerable potential in the future… at present has limited function and definitely needs more technical improvement.”

– Radiology Nov 1988; 169: 580

• “… voice recognition systems are currently not ready for prime time.”

– JACR 2007; 4: 667

• “Has considerable potential in the future… at present has limited function and definitely needs more technical improvement.”

– Radiology Nov 1988; 169: 580

• “… voice recognition systems are currently not ready for prime time.”

– JACR 2007; 4: 667

VR – two decades of controversy

• “Speech recognition systems are used today in more than 1,000 radiology departments and are experiencing a growth rate typical of modern enabling technology.”

– JACR 2007; 4:670

• “Speech recognition systems are used today in more than 1,000 radiology departments and are experiencing a growth rate typical of modern enabling technology.”

– JACR 2007; 4:670

History of VR – a quarter century + of progress despite persistent

controversy

Evolution of Radiology reports

• Paper reports Electronic reports• Paper reports Electronic reports

Transcriptionist model – 1

• Tapes– Batch transcription– Batch correction,

signature

• Tapes– Batch transcription– Batch correction,

signature

Transcriptionist model – 2

• Digital transcription pool– In-line transcription– Batch correction,

signature

• Digital transcription pool– In-line transcription– Batch correction,

signature

Computer data acquisition systems

• Mark-sense forms

• GE RAPORT– AJR 1977; 128: 825

• Mark-sense forms

• GE RAPORT– AJR 1977; 128: 825

Computer data acquisition systems

• Microcomputers – CLIP Harvard

– Numeric codes for reporting– Radiology 1979; 133: 349

– Recognition of spoken numeric codes– Radiology 1981; 138: 585

• Microcomputers – CLIP Harvard

– Numeric codes for reporting– Radiology 1979; 133: 349

– Recognition of spoken numeric codes– Radiology 1981; 138: 585

True VR – Kurzweil system 1987

• Reported by several New England hospitals (including Boston VAMC)

• 1,000 word lexicon

• 5 sections by anatomy or subspecialty

– Radiology 1987; 164: 569.

• Reported by several New England hospitals (including Boston VAMC)

• 1,000 word lexicon

• 5 sections by anatomy or subspecialty

– Radiology 1987; 164: 569.

True VR – Kurzweil system 1987

• Able to dictate a report 88% of the time– 12% beyond scope of lexicon

• Use of macros

• Dictation time 20% longer

• Able to dictate a report 88% of the time– 12% beyond scope of lexicon

• Use of macros

• Dictation time 20% longer

True VR – Kurzweil system 1987

• Drawbacks

– Time and attention diverted from film analysis towards interaction with a monitor

– Increased dictation time proportional to degree of abnormality on film

– Problems with background noise– Problems with repeated interruptions– “Has considerable potential in the future… at

present has limited function and definitely needs more technical improvement.”

» Radiology Nov 1988; 169: 580

• Drawbacks

– Time and attention diverted from film analysis towards interaction with a monitor

– Increased dictation time proportional to degree of abnormality on film

– Problems with background noise– Problems with repeated interruptions– “Has considerable potential in the future… at

present has limited function and definitely needs more technical improvement.”

» Radiology Nov 1988; 169: 580

VR – state of the art 1999

– Error rate 30%

– Misrecognition of words

– Increased dictation timeRadioGraphics 1999; 19: 2.

– Error rate 30%

– Misrecognition of words

– Increased dictation timeRadioGraphics 1999; 19: 2.

VR – today

• Web architecture• Integration with

PACS– Improved efficiency– Decreased errors

• Improved recognition rates

• Decreased turnaround time

• Web architecture• Integration with

PACS– Improved efficiency– Decreased errors

• Improved recognition rates

• Decreased turnaround time

VR case study: Saint Louis VAMC

VR drivers

• PACS implementation– Fewer lost films – More reports

required

• PACS implementation– Fewer lost films – More reports

required

VR drivers

• CPRS implementation

• Universal availability of patient chart

• Clinical demand for faster reports

• CPRS implementation

• Universal availability of patient chart

• Clinical demand for faster reports

VR drivers

• Problem of preliminary reports– Clinical demand– Error correction– Legal issues

• Problem of preliminary reports– Clinical demand– Error correction– Legal issues

VR drivers

• Transcription problems– New contractor (low

bidder)– Cut and paste

errors– Variable

transcriptionist quality

• Transcription problems– New contractor (low

bidder)– Cut and paste

errors– Variable

transcriptionist quality

VR drivers

• Turn around time mandate– 90% completion in 48 hours

successful• 95% completion in 48 hours excellent

– Actual far less

• Turn around time mandate– 90% completion in 48 hours

successful• 95% completion in 48 hours excellent

– Actual far less

Analysis of options

• Hire more Radiologists– Full-time– Part-time

• Retired Radiologists• Fellows

– Recruiting difficulties

• Pay• Vacation

• Hire more Radiologists– Full-time– Part-time

• Retired Radiologists• Fellows

– Recruiting difficulties

• Pay• Vacation

Analysis of options

• Improve efficiency of reporting cycle

VR

• Improve efficiency of reporting cycle

VR

Proposal for VR system

• Strong administration support

• Support for VISN-wide solution– Some centers opted out

• Strong administration support

• Support for VISN-wide solution– Some centers opted out

System evaluation

• Radiologist input• Administration

– ADPAC– PACS coordinator– IT

• Literature review

• Radiologist input• Administration

– ADPAC– PACS coordinator– IT

• Literature review

System evaluation

• Vendor demonstrations– Radiologist trials

• Evaluation of administrator functions

• Vendor demonstrations– Radiologist trials

• Evaluation of administrator functions

Survey existing users

• Most sites only have experience with one system

• Hard to get good comparisons

• Your mileage may vary– Differences in

technical, administrative support for system

• Most sites only have experience with one system

• Hard to get good comparisons

• Your mileage may vary– Differences in

technical, administrative support for system

License issues

• Per unique user• Per workstation• Simultaneous users

vs. individual user

• Per unique user• Per workstation• Simultaneous users

vs. individual user

Vendor recommendation and selection

Planning

• Documentation review

• Site planning

• Documentation review

• Site planning

Results

• Report turnaround 90 – 95 % within 48 hours

• Cost savings

• Report turnaround 90 – 95 % within 48 hours

• Cost savings

VR implementation: lessons learned

• Plan, plan, plan• Plan, plan, plan

Lessons learned – project team

• Identify members– PACS administrator– Transcription

administrator– Editors– IT– Radiologist

• Dedication essential• Time consuming• Work closely with vendor• Read documentation

closely

• Identify members– PACS administrator– Transcription

administrator– Editors– IT– Radiologist

• Dedication essential• Time consuming• Work closely with vendor• Read documentation

closely

Lessons learned – conference calls

• Weekly calls

• Need everyone involved– IT– Administrators– Editors– Radiologist– Vendor

• Weekly calls

• Need everyone involved– IT– Administrators– Editors– Radiologist– Vendor

Lessons learned – conference calls

• Write questions in advance

• Keep minutes– Serves as a

resource– Complex project,

can’t remember everything

– Document to prevent misunderstandings

• Write questions in advance

• Keep minutes– Serves as a

resource– Complex project,

can’t remember everything

– Document to prevent misunderstandings

Lessons learned – installation issues

• Administrator training critical

• Get administrator manuals before vendor rep shows up

• Write questions in advance

• Take notes

• Administrator training critical

• Get administrator manuals before vendor rep shows up

• Write questions in advance

• Take notes

Lessons learned – test, test, test

• Test everything – don’t even think of implementation until this is done

• Vendor supplied checklist

• Test everything – don’t even think of implementation until this is done

• Vendor supplied checklist

Lessons learned – test, test, test

• Test system and test accounts

• Register procedure names and CPT codes

• Enter orders into Vista– Check request entry

into VR system

• Test system and test accounts

• Register procedure names and CPT codes

• Enter orders into Vista– Check request entry

into VR system

Lessons learned – test, test, test

• Dictate test reports– Check for proper

upload– Test addendums

and corrections– Input every type of

diagnostic code– Check parent and

descendants

• Dictate test reports– Check for proper

upload– Test addendums

and corrections– Input every type of

diagnostic code– Check parent and

descendants

Lessons learned – test, test, test

• Change orders• Minimum of 100 test patients• Test every Radiologist

– Include residents• Test sending to editor• Test telephony

• Change orders• Minimum of 100 test patients• Test every Radiologist

– Include residents• Test sending to editor• Test telephony

Lessons learned – Radiologist champion

• Change resistance• Promote system,

convince others that this is way to go

• Upfront buy-in from Radiologists

• Must see as improving patient care– vs. mandate from

administration

• Change resistance• Promote system,

convince others that this is way to go

• Upfront buy-in from Radiologists

• Must see as improving patient care– vs. mandate from

administration

Lessons learned – Radiologist champion

• Overcome objections– “I’m a physician not a transcriptionist!”

• Help others as one professional to another– Keep people going thru rough spots

• Need close communication with remainder of implementation team– Get feedback

• Overcome objections– “I’m a physician not a transcriptionist!”

• Help others as one professional to another– Keep people going thru rough spots

• Need close communication with remainder of implementation team– Get feedback

Lessons learned – Radiologist training

• Radiologists that have problems generally did not get good training

• Radiologists that have problems generally did not get good training

Lessons learned – Radiologist training

• Must have training schedule for every Radiologist

• Everyone has dedicated blocks for training, including follow-up– Minimum 4 hours with

trainer• Some may need more

attention

– Follow-up session

• Must have training schedule for every Radiologist

• Everyone has dedicated blocks for training, including follow-up– Minimum 4 hours with

trainer• Some may need more

attention

– Follow-up session

Lessons learned – Radiologist training

• Once trained, go cold turkey• Continuing support• Dealing with non-native English

speakers• Dealing with poor dictation styles• Refresher training

• Once trained, go cold turkey• Continuing support• Dealing with non-native English

speakers• Dealing with poor dictation styles• Refresher training

Lessons learned – site trainer training

• Must learn to train new users

• Critical if residents involver

• Individual training• Sit in on user

training sessions

• Must learn to train new users

• Critical if residents involver

• Individual training• Sit in on user

training sessions

Lessons learned – continual QC

• Continual effort and vital for long-term success

• Test plan• Test telephony• Dummy orders uploading

• Continual effort and vital for long-term success

• Test plan• Test telephony• Dummy orders uploading

Lessons learned – continual QC

• Pull real reports and monitor for errors

• Intervention if needed

• Retraining of dictator

• Rebuild voice model

• Pull real reports and monitor for errors

• Intervention if needed

• Retraining of dictator

• Rebuild voice model

Lessons learned – continual QC

• Look in CPRS– Report text ok– E-signature ok– Diagnostic codes

• Look in CPRS– Report text ok– E-signature ok– Diagnostic codes

Lessons learned – continual QC

• Monitor continually– Uploads– Orphan dictations– Exams without

reports

• Monitor continually– Uploads– Orphan dictations– Exams without

reports

Lessons learned – continual QC

• Need support contract

• Keep contacts handy

• Know who to call

• Need support contract

• Keep contacts handy

• Know who to call

Lessons learned – trouble log

• Take notes for every trouble call to vendor

• Resource to fix problems on your own– Record

• Day• Ticket #• Who spoke to• Problem• How it was resolved• Note recurring problem

• Take notes for every trouble call to vendor

• Resource to fix problems on your own– Record

• Day• Ticket #• Who spoke to• Problem• How it was resolved• Note recurring problem

Lessons learned – backup plan

• Backup VR server?

• Utilize another transcription contract?

• Other medical center?

• Backup VR server?

• Utilize another transcription contract?

• Other medical center?

PACS setup and reporting with VR

• Kim Wilson MD– Tucson VAMC

• Kim Wilson MD– Tucson VAMC

Radiologist workflow: practical points

Goals

• Increase dictation efficiency

• Maximize eyes on image

• Increase dictation efficiency

• Maximize eyes on image

Transcription models – read, edit, done

• Highly recommended

• Minimize turnaround• Make corrections

while image is in front of you

• Once it’s gone you don’t have to deal with it again

• Highly recommended

• Minimize turnaround• Make corrections

while image is in front of you

• Once it’s gone you don’t have to deal with it again

Transcription models – batch correct, sign

• Most efficient work flow state?

• Longer turnaround

• Error correction more difficult– Right – left– Dates

• Most efficient work flow state?

• Longer turnaround

• Error correction more difficult– Right – left– Dates

Transcription models – editor• Not recommended

• Transforms transcriptionist model correctionist

• Inefficient, expensive• Maximum turnaround

time• Must remember to

correct and sign reports

• Editor errors• When is it helpful?

– Non-native English speakers?

– Poor dictation technique

• Not recommended

• Transforms transcriptionist model correctionist

• Inefficient, expensive• Maximum turnaround

time• Must remember to

correct and sign reports

• Editor errors• When is it helpful?

– Non-native English speakers?

– Poor dictation technique

Transcription styles

• Free dictation

• Templates and macros

• Free dictation

• Templates and macros

Free dictation

• Advantage– Keeps eyes on

image

• Disadvantage– More time with

editing and corrections

• Advantage– Keeps eyes on

image

• Disadvantage– More time with

editing and corrections

Templates – advantages

• Improved time savings

• Improved report accuracy

• Consistent report structure– Personally– Across department– Need agreement among radiologists

• Facilitates structured reporting– BIRADS

• Improved time savings

• Improved report accuracy

• Consistent report structure– Personally– Across department– Need agreement among radiologists

• Facilitates structured reporting– BIRADS

Templates – disadvantages

• Takes eyes of the image

• May forget to delete non-relevant text

• Takes eyes of the image

• May forget to delete non-relevant text

Templates

• Especially useful for repetitive boilerplate– Biopsy– Angiography

• Especially useful for repetitive boilerplate– Biopsy– Angiography

• The patient was placed on the CT table in [<supine> ] position.

• Initial scans were obtained to localize the [ ].

• An appropriate site at the [ ] was marked.

• The patient was prepped and draped in the usual sterile manner. Local anesthesia was achieved with infiltration of 1% Xylocaine.

• The patient was placed on the CT table in [<supine> ] position.

• Initial scans were obtained to localize the [ ].

• An appropriate site at the [ ] was marked.

• The patient was prepped and draped in the usual sterile manner. Local anesthesia was achieved with infiltration of 1% Xylocaine.

Template approaches

• Few general reports– Fill in the blanks– Default fill in the

blanks

• Many specific reports

• Few general reports– Fill in the blanks– Default fill in the

blanks

• Many specific reports

• Case [ ]. [ ]• There is no evidence

of fracture, dislocation, or bony destruction.

• [<The joint spaces are within the limits of normal.>]

• [ < >]• Impression:• [<Negative

examination.>]

• Case [ ]. [ ]• There is no evidence

of fracture, dislocation, or bony destruction.

• [<The joint spaces are within the limits of normal.>]

• [ < >]• Impression:• [<Negative

examination.>]

Templates– itemized reports

– Lungs: [<normal.>]– Pleura: [<normal.>]– Mediastinum [<normal.>]– Hila: [<normal.>]– Other: [< >]– Comparison: [<None.>]

– Impression: [<normal>]

– Lungs: [<normal.>]– Pleura: [<normal.>]– Mediastinum [<normal.>]– Hila: [<normal.>]– Other: [< >]– Comparison: [<None.>]

– Impression: [<normal>]

Template tricks

• Standardize template naming convention

• Modality body part side, technique

• Standardize template naming convention

• Modality body part side, technique

Template tricks

• Make template easy to change on the fly

• Liberal use of paragraphs

• Make template easy to change on the fly

• Liberal use of paragraphs

• Case [ ].• Ultrasound abdominal aorta.

• Real-time ultrasound examination of the abdominal aorta was obtained in transverse and longitudinal projections.

• The patient [<does not have an>] abdominal aortic aneurysm.

• The abdominal aorta measures [ ] cm in maximal diameter.

• [< >]

• Impression:

• [<The patient does not have an abdominal aortic aneurysm.>]

• Case [ ].• Ultrasound abdominal aorta.

• Real-time ultrasound examination of the abdominal aorta was obtained in transverse and longitudinal projections.

• The patient [<does not have an>] abdominal aortic aneurysm.

• The abdominal aorta measures [ ] cm in maximal diameter.

• [< >]

• Impression:

• [<The patient does not have an abdominal aortic aneurysm.>]

Dictation technique

• Fast ok• Must be clear and

distinct– Think before

speaking– Know what you

want to say– No filler sounds

• Fast ok• Must be clear and

distinct– Think before

speaking– Know what you

want to say– No filler sounds

Dictation technique

• Speak in phrases– Get a flow– Correct in phrases rather than individual

words

• Use complete sentences• Use paragraphs liberally

• Speak in phrases– Get a flow– Correct in phrases rather than individual

words

• Use complete sentences• Use paragraphs liberally

Dictation technique

• Consistent style

• Keep reports short

• Don’t number items in impression

• Consistent style

• Keep reports short

• Don’t number items in impression

Dictation technique

• Dictate – then correct– Keep eyes on image

• Read and correct reports carefully before signing

• Dictate – then correct– Keep eyes on image

• Read and correct reports carefully before signing

Microphones

• Proper location

• Headsets?

• Proper location

• Headsets?

Environment

• Noise control

• Bullpen disruption

• Noise control

• Bullpen disruption

Monitor layout

• Open window in admin monitor– Don’t continually

check transcription– Dictate then edit

• Pop-up in admin monitor

• Open window in admin monitor– Don’t continually

check transcription– Dictate then edit

• Pop-up in admin monitor

Monitor layout

• Separate monitor?– VR– CPRS– Internet window –

Google– Decision support

software?– Teaching file

software?

• Separate monitor?– VR– CPRS– Internet window –

Google– Decision support

software?– Teaching file

software?

Training for problem words

• Case number December

• Pulmonary bony

• Adrenal no renal

• Case number December

• Pulmonary bony

• Adrenal no renal

Gotchas

• Impression:

• Dictate case number in every report– Troubleshooting

• Impression:

• Dictate case number in every report– Troubleshooting

Gotchas

• How reports look in VR may not be how report looks in PACS, Vista, or CPRS– Line spacing– New lines vs. paragraphs

• How reports look in VR may not be how report looks in PACS, Vista, or CPRS– Line spacing– New lines vs. paragraphs

Residents

• Pre-dictation by resident

• Make corrections and finalize report at time of checking

• Easy sign-off by attending

• Drawbacks – templating– May not learn elements of

a good report

• Pre-dictation by resident

• Make corrections and finalize report at time of checking

• Easy sign-off by attending

• Drawbacks – templating– May not learn elements of

a good report

Success rules for VR

• You must want system to work

• Training the VR vs. VR training you

• Rule of thirds

• You must want system to work

• Training the VR vs. VR training you

• Rule of thirds

Future development of VR

Improved recognition engines

• Better accent recognition

• Better recognition of small words

• Better accent recognition

• Better recognition of small words

Improved integration of PACS, HIS-RIS

• Too easy to mark case as read when not• Too easy to mark case unread when is

read• Too easy to hang up report

– Impression:

• Too easy to forget to sign off on report

• Too easy to mark case as read when not• Too easy to mark case unread when is

read• Too easy to hang up report

– Impression:

• Too easy to forget to sign off on report

Improved grammar checking

• There• Their• They’re

• Two• Too• To

• Capitalization

• There• Their• They’re

• Two• Too• To

• Capitalization

Structured reporting

• Standard lexicons

• Universal framework for reports– Improve readability– Minimize style

variation between Radiologists

– Data mining

• BIRADS

• Standard lexicons

• Universal framework for reports– Improve readability– Minimize style

variation between Radiologists

– Data mining

• BIRADS

Seamless integration of communication

• Clinical alerts

• Paging for critical findings

• Feedback to technologist, QA supervisor

• Clinical alerts

• Paging for critical findings

• Feedback to technologist, QA supervisor

Ultimate VR goal: universal recognition

Outlook Mailgroup

• VHA Radiology Voice Recognition

• VHA Radiology Voice Recognition