Implementing a Client Feedback System to Improve the Quality of ...

Q U A L I T Y

A S S U R A N C E

P R O J E C T

Center for Human Services • 7200 Wisconsin Avenue, Suite 600 • Bethesda, MD 20814-4811 • USA • www.qaproject.org

O P E R A T I O N S R E S E A R C H R E S U L T S

Implementing a Client FeedbackSystem to Improve the

Quality of NGO Healthcare Services in Peru

The Quality Assurance (QA) Project is funded by the U.S. Agency for International Development (USAID),under Contract Number HRN-C-00-96-90013. The QA Project serves countries eligible for USAIDassistance, USAID Missions and Bureaus, and other agencies and nongovernmental organizations thatcooperate with USAID. The QA Project team consists of prime contractor Center for Human Services(CHS), Joint Commission Resources, Inc. (JCI), Johns Hopkins University School of Hygiene and PublicHealth (JHSPH), Johns Hopkins Center for Communication Programs (JHU/CCP), and the Johns HopkinsProgram for International Education in Reproductive Health (JHPIEGO). It provides comprehensive,leading-edge technical expertise in the design, management, and implementation of quality assuranceprograms in developing countries. CHS, the nonprofit affiliate of University Research Co., LLC (URC),provides technical assistance and research for the design, management, improvement, and monitoring ofhealthcare systems and service delivery in over 30 countries.

O P E R A T I O N S R E S E A R C H R E S U L T S

Abstract

Significant improvements tohealthcare services are possible withan understanding of clients’perspectives, but those perspec-tives often go unexpressed or areexpressed without detail throughverbal complaints or complaintsdropped in suggestion boxes. Exitinterviews and focus groups areamong several methods for collect-ing detailed information from clients,but little research has been done totest those methods in developingcountries.

The Quality Assurance (QA) Projectinvestigated six methods forcollecting client feedback at twohealthcare clinics in Chiclayo, Peru,from September 1998 to April 1999.This report summarizes the resultsof that study, presenting both adescription of the informationcollected and a comparison of sixdata collection methods. Informationis also presented on how managersinvolved in the study used the datato improve clinic services. Thereport concludes with lessonslearned and guidance on how toimprove client feedback systems.

The study site was the Max SaludInstitute for High Quality HealthCare, started in 1994 with fundingfrom the U.S. Agency for Interna-tional Development (USAID). MaxSalud is a private, nonprofit organi-zation whose clients chose betweenusing its services and nearby publicclinics. At the time of the study, MaxSalud provided a broad range of

Table of Contents

I. Introduction ...................................................................................................... 1

II. Research Setting .............................................................................................. 2

III. Research Design and Methodology .................................................................. 2A. Data collection ............................................................................................. 2B. Data analysis ............................................................................................... 4C. Providing client satisfaction results to Max Salud ........................................ 5

IV. Findings on Client Satisfaction.......................................................................... 5A. Effectiveness ................................................................................................ 5B. Efficiency ..................................................................................................... 5C. Technical competence ................................................................................. 6D. Interpersonal relations.................................................................................. 7E. Access to services ....................................................................................... 7F. Safety .......................................................................................................... 8G. Continuity .................................................................................................... 8H. Physical aspects .......................................................................................... 8

V. Findings: Analysis of Client Feedback System................................................ 10A. Advantages and Disadvantages of the Data Collection Methods .............. 11

VI. Findings: Use of Client Feedback at Max Salud ............................................. 11A. Quality committee of the management support unit .................................. 11B. Balta quality committee ............................................................................. 11C. Urrunaga quality committee ....................................................................... 12

VII. Lessons Learned: Improving Client Feedback ................................................ 13

VIII. Conclusion ..................................................................................................... 14

References ..................................................................................................... 15

Implementing a Client Feedback SystemTo Improve the Quality of NGOHealthcare Services in Peru

continued on next page

Acknowledgement

This paper was written by Diana Santillán, principal investigator, QualityAssurance (QA) Project, and Maria Elena Figueroa, Johns HopkinsUniversity Center for Communications Programs (JHU/CCP). The authorsgratefully acknowledge the collaboration and support of the Max SaludInstitute for High Quality Health Care, including the valuable input andassistance of Filiberto Hernández, Project Director; Luis Castañeda,Medical Services Director; Oswaldo Sierra, Training and Information,Education, and Communication (IEC) Advisor; Sonia Tamayo, CommunityOutreach Advisor; and Isidoro Benites, Information System Specialist.

Technical contributions in the study design and writing were provided byMax Tello, Marcela Tapia, Katy Cáceres, Gary Lewis, and Patricia Poppe(JHU/CCP); and Paula Tavrow, Paul Richardson, and Bart Burkhalter (QAProject). The study would not have been possible without the dedicationof Silvia Arrascue and Fátima Linares, who participated as data collectorsand research assistants. Raul Alberto Robles and Victor Hugo Vargas alsoprovided important assistance to the research team.

Recommended citation

Santillán, D., and M. E. Figueroa. 2001. Implementing a client feedbacksystem to improve the quality of NGO healthcare services in Peru.Operations Research Results (2)2. Bethesda, MD: Published for theUnited States Agency for International Development (USAID) by theQuality Assurance Project (QA Project).

About this series

The Operations Research Results series presents the results of the QAProject country or area research to encourage discussion and commentwithin the international development community. To obtain the fullresearch report of this study, please contact [email protected].

health services to 20,000 low- tomiddle-income people through twoclinics supported by a centralmanagement unit.

Max Salud wanted the clientfeedback system in order to collectclients’ perspectives of services andto convey those perspectives insummary fashion to its qualitycommittees, which would then usethe information for quality improve-ment. Exit interviews, follow-upvisits, focus group discussions,interviews with discontinued clients,suggestion boxes, and communitymeetings were the intended datacollection methods, although thecommunity meetings, ongoing at thestart of the study, proved to havebeen structured in such a way thatelicited little client feedback.

The data from the client feedbacksystem indicated high client satisfac-tion with the quality of services,especially the friendly personnel,clean and pleasant settings, andprompt service. Quality improve-ments resulting from the studyincluded improving the responsetime to client complaints, sensitizingclinic personnel to clients’ concerns,and reducing waiting times.

The validity, utility, and feasibility/costof the different methods variedconsiderably. For example, exitinterviews were very feasible andprovided quantitative data valued byquality committees, but theirstructured format limited clients’expressions of dissatisfaction. Focusgroups yielded rich, detailedinformation, but were expensive andtime consuming.

Abstract Continued

Client Feedback in Peru ■ 1

Diana Santillán and Maria Elena Figueroa

Acronyms and Abbreviations

IEC Information, education,and communication

JHU/CCP Johns HopkinsUniversity Center forCommunications Programs

MSU Management Support Unit

NGO Nongovernmental organization

QA Quality assurance

QI Quality improvement

URC-CHS University ResearchCorporation-Center forHuman Services

USAID United States Agency forInternational Development

Implementing a Client Feedback SystemTo Improve the Quality of NGOHealthcare Services in Peru

I. Introduction

When clients experience high-qualityhealthcare and their health improves,they generally feel more satisfied.Interestingly, scientific evidenceindicates the reverse is also true:increased client satisfaction can leadto better health outcomes. Satisfiedclients are more likely to comply withtreatment and advice and to return foradditional care as necessary (Aharonyand Strasser 1993; Lochman 1983).They are also more willing to pay forservices (Scott and Smith 1994),thereby increasing revenue forhealthcare, an important element ofhealth sector reform. Client satisfac-tion has been characterized as both arational evaluation of, as well as anemotional reaction to, the quality ofcare, that is, to the structure, process,and outcome of services (Pascoe1983; Donabedian 1988). While thevalue of raising client satisfaction iswell recognized, the optimal methodsof collecting and using data remainunknown.

Although the literature on clientsatisfaction is extensive (Cleary andMcNeil 1988; Pascoe 1983), it is

mostly based on North American andEuropean studies. Client satisfactionstudies from developing countries arerare (Tavrow 1997). Moreover, fewclient satisfaction studies haveassessed the use of client satisfactiondata in quality improvement (QI)activities (Aharony and Strasser 1993).A study by Arnetz and Arnetz (1996) isa notable exception. A regionalSwedish hospital staff used the resultsof a client satisfaction questionnaire toplan QI activities; client satisfactionwas reassessed after the activities andshowed significantly higher ratings(see also Shelton 2000; Williams,Schutt-Ainé, and Cuca 2000).

This report summarizes the findings ofa study by the Quality Assurance (QA)Project, a nonprofit organizationbased in Bethesda, MD, and man-aged by the University ResearchCorporation-Center for HumanServices (URC-CHS). The study testeda client feedback system at the MaxSalud Institute for High Quality HealthCare, a nonprofit healthcare organiza-tion in Chiclayo, Peru. At the time ofthe study, Max Salud provided abroad range of healthcare services toabout 20,000 low- to middle-incomepeople. The study objectives were togather client satisfaction data, analyzethe data collection methods, andassess the use of the client satisfac-tion data by quality assurance (QA)committees. The client feedbacksystem, implemented at two MaxSalud clinics from May 1998 to May1999, employed exit interviews,

2 ■ Client Feedback in Peru

follow-up visits, focus groups,household interviews of clients whohad stopped using Max Saludservices, suggestion boxes, andcommunity meetings.

This report provides information on allthree objectives, detailing the validity,utility, feasibility, and cost of thedifferent data collection methods. Italso describes the data and how theywere used. Last, this report hasrecommendations for making clientfeedback an integral part of improvingthe quality of healthcare delivery indeveloping countries.

II. Research Setting

The Max Salud Institute for HighQuality Health Care was founded in1994 as part of the StrengtheningHealth Institutions Project/NorthernComponent, funded by the UnitedStates Agency for InternationalDevelopment (USAID) and imple-mented by University Research Co.,LLC (URC) and Clapp & Mayne. MaxSalud is located in Chiclayo, a coastalprovince in Peru. Max Salud decidedto undertake the study after recogniz-ing the connection between satisfyingclients and financial sustainability.

The Max Salud Management SupportUnit (MSU), comprised of directors,advisors, and administrative staff,provided technical and logisticalassistance for the operation of theclinics. Quality committees were partof the organizational structure of MaxSalud, both at the central and cliniclevel. The MSU quality committeeincluded the project director, executivedirector, medical services director,clinic operations director, director offinance and administration, logisticsand supplies director, and the trainingand IEC (information, education, andcommunication) advisor. Quality

committees at each clinic included theclinic director, a doctor, a nurse, acertified midwife, an administrativeassistant, and a social worker. Thepurpose of the committees, which metmonthly, was to monitor, sustain, andimprove the quality of Max Saludhealthcare services.

The study implemented a clientfeedback system at both Max Saludclinics: Balta and Urrunaga. The BaltaClinic was situated in a busy, commer-cial downtown area. This small clinicoffered primary healthcare services,24-hour emergency services, andsome medical specialties, such aspediatrics and gynecology. Approxi-mately 12,500 people lived in itscatchment area, but clients came fromvarious parts of the city, often incorpo-rating a medical visit into a day oferrands downtown. The UrrunagaClinic, located in a peri-urban commu-nity, served a lower-income popula-tion. It was open only half a day, sixdays a week, and did not providemedical specialty services. However, itengaged in more community outreachand health promotion activities thanBalta, with the help of about 30volunteer health promoters. Thecatchment area population forUrrunaga was 7,500.

III. Research Design andMethodology

The study implemented and continu-ously refined a client feedback systemthat included various methods tocollect and analyze client satisfactiondata and provide feedback to thequality committees. The researchteam—a principal investigator and twodata collectors—performed twowaves of data collection. After eachwave, the team analyzed and summa-rized the client satisfaction results,

and presented them to the qualitycommittees at the MSU and clinics.During these feedback meetings, thequality committees discussedproblems identified by clients andpossible solutions to improve servicequality, such as process improve-ments, rapid management responses,or individual follow-up with specificclients. Figure 1 illustrates the systemimplemented by the study.

A. Data collectionAs noted above, two waves of eachdata collection method were con-ducted, as summarized in Table 1.

Exit interviewsExit interviews of clients as they areleaving a clinic after using its servicesare frequently used to capture clientperceptions of the quality of care.During each wave, data collectorscompleted at least 80 exit interviewsat each clinic, on different days and atdifferent hours to ensure a mix ofclients. Only a few clients declined theinterview.

At the Balta Clinic, data collectorsinterviewed clients while sitting on thebenches of the waiting area or whilestanding just outside the entrance tothe clinic. At the Urrunaga Clinic, alarge room available for communitymeetings and near the entrance wasused for these interviews. A question-naire was used and had mostly open-ended questions with possibleanswers for each question. Theseanswers were not read aloud; the datacollectors simply marked the appropri-ate choice depending on the clients’responses. Interviews took about 10minutes each.

For purposes of the follow-up visitsthat would take place after the exitinterviews, data collectors askedclients for permission to visit them in


the future. If the clients agreed,addresses and convenient times wererecorded.

Follow-up visitsFollow-up visits were used to controlfor possible “courtesy bias”1 in the exitinterviews. For each wave, eightclients (10 percent) who had partici-pated in exit interviews received afollow-up visit. Follow-up visits weresemi-structured interviews held withina week after a client’s visit to theclinic. The follow-up guide covered thesame topics as the exit interview.These interviews occurred at clients’homes to allow clients to elaboratewithout clinic staff nearby. Thesesessions collected clients’ reflectionsafter they had an opportunity toconsider their clinic experience andallowed clients to report on theeffectiveness of the treatment theyhad received. With client permission,data collectors tape-recorded eachfollow-up visit, which lasted approxi-mately 20 minutes.

Although initially intended as a validitycheck for the exit interviews, anotheruseful application of the follow-upvisits emerged. Data collectorspurposively selected a high proportionof clients who expressed dissatisfac-tion during the exit interviews forfollow-up visits. This selection processgenerated more detailed informationabout negative experiences from thesmall subset of dissatisfied clients.Other clients for follow-up visits wereselected at random.

Focus groupsFocus group discussions broughttogether groups of clients to discusstheir experience with the clinic’sservices. Eight focus groups, stratified

Figure 1

Client Feedback System

Note: “Household Interviews” refers to interviews with discontinued clients at their homes.“Follow-Up Interviews” refers to interviews with recent clients who had had exit interviews.“Follow-Up with Individuals” refers to Max Salud communications with clients who had“expressed dissatisfaction during earlier steps in the data collection process.

FocusGroup

Discussions

ExitInterviews

SuggestionBoxes

CommunityMeetings

Follow-UpVisits

HouseholdInterviews

Data Collectors

Quality Committees

Service Quality

Rapid ManagementResponses

ProcessImprovements

Follow-Up withIndividuals

Table 1

Data Collection Schedule and Sample Sizes

Timeframe ExitInterviews

Follow-UpInterviews

Focus GroupDicsussions

HouseholdInterviews

CommunityMeetings

SuggestionBoxes

Sept.–

Oct. 1998

Nov. 1998 –Jan. 1999

1 for Balta

1 for Urrunaga

83 for Balta

80 for Urrunaga

8 per clinic 1 for Balta 9 returns from Balta

4 returns fromUrrunaga

1 per clinic

10 per clinic 13 returns from Balta

12 returns fromUrrunaga

1 per clinic

Feb. 1999 80 per clinic 8 per clinic (Redesign of form) 1 per clinic

Mar. –April 1999

2 for Balta

3 for Urrunaga

10 per clinic (Validation and launchof new form)

1 per clinic

Total 323 32 8 40 38 8

1 Courtesy bias is when people give inaccurate information because of a desire to please others, to avoid insulting someone, or to preventembarrassment.


by sex and age, were held (four perclinic). Participants were randomlyselected using the Max Salud Man-agement Information System. Basedon the pre-test experience with focusgroups, data collectors personallydistributed invitations to about 30clients to obtain six to eight partici-pants for each group.2

Data collectors acted as facilitator andnote-taker, and a focus group guidehelped to direct the discussions,which were tape-recorded with theclients’ permission. At the end of eachfocus group discussion, which lastedabout two hours, refreshments wereserved, and each participant receiveda small gift and money for transporta-tion.

Household interviewsIdentifying the sources of clientdissatisfaction from clients whocontinue to use the facility excludesthose who became so dissatisfiedwith services that they stoppedseeking services there. In order toinclude clients whose dissatisfactionhad achieved a level sufficient tocause them to discontinue use of MaxSalud services, the research teaminterviewed “discontinued clients.”They were defined as clients who hadvisited the clinic a few times and hadnot returned for over a year. Theexpectation was that some of theseclients had discontinued use becauseof dissatisfaction. Data collectors usedMax Salud client lists to randomlyselect discontinued clients, excludingthose who lived more than half anhour away (roughly 10 percent).

For each wave, the data collectorsused a household interview guide tointerview 10 clients from each clinic.

Interviews averaged 25 minutes inlength and were tape-recorded withthe clients’ permission.

Suggestion boxesSuggestion boxes are quite commonin health facilities in Peru and wereused at all Max Salud clinics beforethe study began. The data collectorscompiled the comments from theseboxes on two occasions to comple-ment the other methods used in thestudy. The comments were written onforms designed by Max Salud, buthad not been collected for severalmonths.

After the second wave, the researchteam redesigned the suggestion boxform in collaboration with the MSUtraining and IEC advisor. The new formincluded illustrations and asked clientsto rank different aspects of the clinicby selecting a smiling, neutral, orfrowning face, a technique intended tospeed up the process of completingthe forms and to accommodateclients with poor literacy. The researchteam tested the new form in a focusgroup and then distributed it to theclinics.

Community meetingsData collectors attended communitymeetings, convened by Max Saludsocial workers at each clinic todiscuss various health topics. Theresearch team attended and observedsome of these meetings in order tounderstand the local context andinterpret clients’ comments. Themeetings also provided a source ofinformation in a more informalenvironment than the other datacollection methods. The formatprovided an opportunity to hearcommunity perspectives on health

issues and might have generateddifferent concerns than would arise informal data collection exercises.

The community meetings were notspecifically part of the study and noguidance or direction was provided tothe meeting facilitators. At eachcommunity meeting, the data collec-tors took notes of their observations.

B. Data analysisData from the exit interviews wereentered into a database developed forthe client feedback system to gener-ate tables summarizing the results.

Data collectors transcribed the tape-recorded follow-up visits, focusgroups, and household interviewsverbatim. Transcriptions were codedfor key comments reflecting satisfac-tion and dissatisfaction using eightquality dimensions of healthcaredelivery (Table 2). Data collectors alsohighlighted clients’ commentsregarding barriers to services,comparison of Max Salud with otherhealth services, and suggestions forimprovement.

Clients’ comments from the sugges-tion box forms were entered into aspreadsheet as well. The communitymeetings were educational sessionsand not interactive enough to renderuseful data about clients’ experiencesin the community, contrary to theresearchers’ prior expectations.Consequently, their content wasneither analyzed nor presented to thequality committees. However, thesemeetings did enlighten the datacollectors about the clinics’ culturalcontext, facilitating data interpretation.

2 This suggests a potential self-selection bias. Project staff suspect that the participants tended to be more dissatisfied than non-participants andthat these participants wanted to voice their complaints.


C. Providing client satisfac-tion results to Max SaludThe research team monitored seriousclient complaints, promptly bringingthem to the attention of the MSUmedical services director to ensure arapid response. The research teampresented the quantitative andqualitative results obtained in eachwave of data collection during threemeetings with the quality committeesof the MSU and clinics. The MSUreceived results for both clinics, whilethe clinic committees received theresults pertaining only to their clinic.The team and committees used thesemeetings to identify problems,opportunities for improvement, andpossible solutions and quality im-provement activities.

IV. Findings on ClientSatisfaction

The study found that Max Saludclients had very positive perceptionsof the services provided by bothclinics. In general, more positive thannegative remarks were obtainedacross all eight quality dimensionsinvestigated. High satisfaction relatedto client-provider relations; cleanlinessand pleasantness of the environment;and efficient, prompt service. Detailsrelating to each of the eight dimen-sions of quality follow, along withcomments from clients.

A. EffectivenessEffectiveness relates to the accuracyof the medical diagnosis and thecapacity of the treatment to cure theillness or, in the case of preventive

DIMENSIONS DESCRIPTION

Effectiveness The degree to which desired results (outcomes) of care are achievedthrough appropriate diagnosis and treatment

Efficiency The ratio of the outputs of services to the associated costs of producingthose services (taking into consideration both material and time resources)

Technical The degree to which tasks carried out by health workers and facilitiescompetence meet expectations of technical quality (according to clinical guidelines)

Interpersonal Level of respect, courtesy, responsiveness, empathy, effective listening,relations and communication between clinic personnel and clients

Access to service The degree to which healthcare services are unrestricted by geographic,economic, social, organizational, or linguistic barriers

Safety The level of trust, confidentiality, and privacy in the services and thedegree to which the risks of injury, infection, or other harmful sideeffects are minimized

Continuity The degree to which consistent and constant care is provided, includingthe value of visiting the same provider and continuing treatment

Physical aspects The physical appearance of the facility and the level of cleanliness,comfort, and amenities offered

(Source: Brown et al. Undated)

Table 2

Quality Dimensions of Health Service Delivery

services, to sustain health. Exitinterviews rarely capture comments oneffectiveness, since clients frequentlydo not know the outcome of their visitwhen they exit. However, qualitativedata from the other methods showthat clients highly value effectiveresponses to their health concerns.Many clients reported being satisfiedby “good medicine,” “accuratediagnoses,” and treatments that“control the illness.”

B. EfficiencyEfficiency refers to clients’ perceptionsof how well personnel performadministrative processes and useresources. Max Salud clients reportedthat the service was expeditious andthat waiting times were reasonable.Few Balta clients (only 10 percent ineach wave) stated upon exit that thewaiting time was too long.

During the first wave of data collectionin Urrunaga, 28 percent of clientsreported that the waiting time was toolong, but for the second wave thispercentage fell to 11 percent. Thisimprovement reflected actions takenby the clinic management to addressa personnel shortage during height-ened demand for services. Takingboth waves of data collection at bothclinics into consideration, more thanhalf (52 percent) of Max Salud clientsreported waiting 15 minutes or less.

“The doctor gave me a prescriptionand now I’m better.”

Urrunaga client, follow-up visit,21-year-old woman

“I arrived sick and left well. Thatsays everything.”

Former Balta client, household interview,18-year-old man


Clients also valued specialistshighly. In fact, when clients haddoubts about the technical compe-tence of providers, they assumedthat they were interns, even thoughall Max Salud staff are fully trainedprofessionals.

“They take too long; they do notrespect appointments. I havewasted time.”

Balta client, follow-up visit,28-year-old man

“People arrived after me and theywere already seeing them.”

Urrunaga client, follow-up visit, 19-year-old woman

“There is no doctor exclusively foremergencies, one needs to wait forhim to come from the regularconsultation room.”

Former Balta client, household interview,38-year-old woman

“Emergency care is supposed to begiven quickly. We waste time if wehave to take him all the way to theBalta Clinic instead.”

Urrunaga client, focus group of 18- o30-year-old women

“They examine you fromthe top of your head toyour toenail.”

Balta client, follow-up visit,23-year-old woman

“The dentist does not make it hurttoo much; he’s careful.”


“There is no pediatrician, onlygeneral medicine; specialists arebetter trained.”

Former Urrunaga client, householdinterview, 24-year-old woman

“There are too many interns, andthey do not know the laboratorywork well.”

Urrunaga client, focus group of womenover 30 years old

Any waiting was unacceptable toclients who came for emergencytreatment. Again, the problem waspoor communication: clients were notmade aware of clinic hours or staffingprocedures for emergency care. SomeUrrunaga clients with urgent problemsfelt they wasted time by going to theclinic, only to find that emergency carewas not available when they needed it.

C. Technical competenceTechnical competence refers to theability and performance of healthproviders as measured againstclinical guidelines. Clients often havedifficulty assessing technicalcompetence, but they value it andrely on certain indications to assessit. When asked what they liked mostabout Max Salud, Balta andUrrunaga clients ranked technicalcompetence third (after friendlypersonnel and clean environment).Many clients recognized that MaxSalud health providers are trained

“It is well organized, one gets theticket and goes to the consultationright away without going fromhere to there, do this or the other;it’s faster.”

Balta client, focus group,women over 30 years old

“It is a good service, the quality andearnestness in their work. It’s fast. Ididn’t have any problems, one givesthe information that they need andstraight ahead.”

Former Urrunaga client, householdinterview, 22-year-old man

The cases when waiting times wereunacceptable related to communica-tion problems or the occasional clientwho became “lost in the shuffle.” Thefact that clients with appointmentstook precedence over walk-insconfused some who did not under-stand why clients were not served inthe order of arrival.

professionals and appreci-ated that they take the timeto examine them carefully.Clients did not want quickcheckups, but rathermeticulous and thoroughones. Rapid service, oftenconsidered a sign ofefficiency, was consideredan indicator of technicalincompetence in theconsultation room. In thecase of dentistry, clientsemphasized the importanceof receiving painless care.


D. Interpersonal relationsInterpersonal relations refers to theverbal and nonverbal communicationsthat clients experience. Interpersonalskills were very valued and appreci-ated by Max Salud clients. Whenasked what they liked most about theclinic, friendly personnel ranked firstfor both clinics’ clients, representing33 percent of clients’ responses in theexit interview. Remarks about positiveinterpersonal relations, such askindness and interest/concern shownby doctors, abounded in the clients’discourse. Clients also highlighted thegood interpersonal relations withnurses and administrative personnel atboth clinics.

average or low, with over halfresponding that they were low. ForUrrunaga, it is important to notethat the data collection coincidedwith the lowering of fees to almostmatch those of public health services.The new rates made Max Salud’sservices much more accessible toUrrunaga clients, who appreciated thespecial rates for their low-incomesector where “there are people whoneed it.”

“The doctor explained very nicely;the nurse is also very friendly; theyare all amiable. The doctor explainedhow things are, in a friendly way.They help us when we arrive in pain;they try to cheer you up.”

Urrunaga client, follow-up visit,33-year-old man

“The gynecologist scares hispatients with impertinent ques-tions. ‘Is your husband young?Does he cheat on you?’ He is veryserious and distant; he has a roughway of saying things.”

Balta client, focus group of18- to 30-year-old women

“The pediatrician was in a badmood; he was a despot; he wasangry. He told me to calm down. Ileft confused; he didn’t explainanything to me.”

Former Balta client, household interview,40-year-old woman

“People are afraid to go becausethere’s too much delinquency.”


“It is in a good place where mostpeople can see it.”

Urrunaga client, focus group, women over 30 years old

“There is no problem, one takes themini-bus, and it drops you rightthere at the clinic.”


“Max Salud prices are now aboutthe same as those in the healthpost, so I decided to come hereinstead.”

Urrunaga client, focus group of18- to 30-year-old women

“Prices are very affordable, themedicine is close to affordable.”

Balta client, focus group ofwomen over 30 years old

“When we have a medical problem,we don’t go because they do notprovide service in the afternoon.”

Urrunaga client, focus group ofwomen over 30 years old

“They are closed on Sundays. Andwhat if there’s an emergency?”


A few Balta clients complained aboutrude treatment from the medicalspecialists, such as the gynecologistand pediatrician. These specialistswere not full-time Max Salud staff and,as such, had not received the sametraining in interpersonal relations andcommunication as full-time staff.Medical specialists attended to clientsonly a few hours a day, and yet thesefew hours were enough to mar thegenerally positive image of Max Saludregarding client-provider interactions.

Service hours were less satisfactoryin Urrunaga, where the clinic isopened only half a day, six days aweek. Many clients felt that they hadno assurance of receiving serviceswhen needed, especially worrisomefor childbirth, which may take placeat night and without warning. BothBalta and Urrunaga clients alsomentioned that they experiencedifficulties when they need clinicservices and do not have anyoneto care for their children.

“They make us feel good; they treatus well, even though I’m verynervous sometimes. Even thesecurity guard greets everyone. I seethat the obstetric nurse is patient.”

Balta client, follow-up visit,16-year-old girl

E. Access to servicesAccess to services refers to clients’ease in visiting the clinic. Exit inter-views indicated that 70 percent ofMax Salud clients experienced nodifficulties. Some clients reported thatthe clinics were located in dangerousareas, but others believe they arestrategically located.

The general consensus was that feelevels were satisfactory. Exit inter-view results showed that more than90 percent of Balta and Urrunagaclients stated that fees were either


One client reported a seriouscomplaint against a provider whohad, either accidentally or intention-ally, placed his hand on her legduring the consultation. Thissituation is either a serious misun-derstanding or inappropriatebehavior that would be a violationof someone’s sense of security.

television. One Urrunaga clientstated that the clinic is a “luxury”for their low-income community.However, for Balta, there were severalcomplaints about the small size of theclinic and congestion in the waitingarea. The top-ranking suggestion forimprovement given by Balta clientswas “larger clinic” (24 percent ofsuggestions). There were also somecomplaints about stock-outs in thepharmacy.

“I felt uncomfortable. I like toconverse so I don’t know if thedentist misinterpreted that or if hewas trying to cross the line or maybeI am mistaken, but he placed hishand on my leg. . . I’m tooembarrassed to return because ofhim. If I do go back, I will go in withmy husband.”

“I have to go back; I still need sometreatments. I would go to MaxSalud always.”


“My daughter liked it. In fact, myhusband took my other daughterthe next day.”


“The clinic entertains one’s sight.”Urrunaga client, focus group

of 18- to 30-year-old men

“Their place is not adequate, thewaiting area is very small.”

Balta client, focus group of menover 30 years old

“They almost don’t have medicinesin the pharmacy. We did not findwhat the doctor prescribed; weneeded to go find it outside.”


F. SafetyThe quality dimension of safety fromthe client’s perspective relates to howsecure clients feel at the clinic. In exitinterviews, 98 percent of Max Saludclients reported that the providersrespected their privacy (e.g., by notletting other people enter the consul-tation room without their permission).Also, clients felt the services were safeand trusted Max Salud providers.

“The truth is that I have faith inthem.”


“It’s tranquil; it’s safe. They inspireconfidence; the instruments arehygienic.”

Urrunaga client, follow-up visit, 20-year-old woman

G. ContinuityContinuity refers to whether clientsobtain consistent care, such as beingable to meet with the same providerevery visit. The results indicated aclear tendency of clients to keep usingMax Salud services. When asked ifthey would return to the clinic forneeded services, virtually all clients (95percent) responded positively in theexit interview. Similarly, 100 percentwould recommend the clinic servicesto others. Overall, 80 percent of MaxSalud clients stated that they wouldlike to see the provider again.

H. Physical aspectsThe physical dimension of qualityrefers to its cleanliness, organization,and ambience. Clients respondedpositively on all of these. “Cleanenvironment” was the second mostfrequent response (14 percent) for theexit interview question, “What do youlike most about this clinic?” One clientstated that the clinic’s cleanlinessmade it “a pleasure to enter, even ifone is not sick.”

Remarks indicating that the facilitieswere “cozy,” “comfortable,” and“organized” also reflected clients’satisfaction with physical aspects ofthe clinic. Clients enjoyed the decora-tion and colors of the clinic (blue andwhite), and amenities such as the


Rau

l Alb

erto

Rob

les


Table 3 continued on page 10

Table 3

Advantages and Disadvantages of Data Collection Methods

Advantages Disadvantages

Exit InterviewsValidity Large, random sample of clients is possible Courtesy bias may result from nearness to clinic staff

Clients recall experience clearly Clients have little time to process experienceCollects clients’ unedited first impressions Clients may answer questions mechanically

Data collectors may ask questions mechanicallyImpersonalClients are often pressed for timeNo time to probe deeplyCollects data only from clients who come to the clinic and receive servicesQuantitative, statistical analysis is “cold,” since it turns clients words andexperiences into numbers

Utility Information is practical, easy to understandData can be used to measure compliance with standards

Feasibility/ Conducted in one place Too many distractions in clinicShort duration Requires a private, comfortable spaceData collectors become familiar with tool due to the repetitive nature of Data entry is repetitive and tediousthis methodEasy to identify participantsLow costRapid data processing if computer database is usedShort time needed for organization/preparation

Follow-Up VisitsValidity Privacy may improve clients’ comfort and openness Data collectors chose clients who are easier to reach (within 30 minutes),

Complements and validates exit interview possibly biasing resultsClients may feel that follow-up is redundant

Utility Can be used to follow up on specific complaints from exit interviewsand suggestion boxMore detailed, in-depth informationEffectiveness of treatment can be assessed after time lapse from clinic visitClients and/or family members may pose new complaintsSeeing or hearing client’s own words helps sensitize clinic personnel

Feasibility/ Access to clients is easier since they have already agreed to be Incorrect addresses deter access to clientsinterviewed (during exit interview) Difficult to follow up on out-of-town clientsAccurate names and addresses are available Unsafe neighborhoods can limit the available hours for interviewsEasy to identify participants Difficult to find male clients at home in some communities duringMore conversational interview with open-ended questions hours available for interviewsData transcription increases data collectors’ familiarity with data Clients who had returned to facility need to be replaced for sample3

and facilitates analysis Transcription is labor-intensiveIdentifying and traveling to clients is time-consuming

Other Transcribing interviews allows data collectors to self-assess their interviewskills

Focus GroupsValidity Group dynamics may cause clients to express themselves more openly Clients’ individual opinions can be influenced by the group’s opinion

Less courtesy bias due to privacy from clinic personnel

Feasibility/ High investment of time and cost: organization, collection, transcription,and analysis

Other Empowers clients by giving them a chance to express themselves publiclyTranscribing data allows data collectors to self-assess their facilitation skills

3 It is important to capture information consistently. If a client attends a focus group and is selected for follow-up but returns to the clinic prior tothat follow-up, the intervening appointment would render the information inconsistent with other members of the sample.

Cost

Cost

Cost


Table 3, continued

Advantages and Disadvantages of Data Collection Methods

Advantages Disadvantages

Household InterviewsValidity Less courtesy bias due to privacy from clinic personnel Clients do not recall experiences clearly

Clients feel comfortable at home and express themselves openly Information may be out-of-dateClients have had time to consider their clinic experience and react Many clients did not stop using the clinic due to quality reasons, so it takes a

lot of interviews to a get significant information from truly dissatisfied clients

Utility Provides specific information on why clients have not returned to the clinic Transcription is time-consumingSeeing or hearing clients’ own words helps sensitize clinic personnel

Feasibility/ Easier to organize than focus groups Incorrect addresses delay the processTranscribing data facilitates analysis Difficult to reach distant clients

Unsafe neighborhoods may limit available hours for interviewsDifficult to find male clients at home in some communities during available hours

Other Clients learn that the clinic cares about them and return theirattention to clinic servicesTranscribing data allows data collectors to assess their interview skills

Suggestion Boxes

Validity Client is assured anonymity if desired Information cannot be verifiedGives clients a way to vent frustrations or express opinions at will Data may be used inappropriately for internal conflicts or attacks on staffCould empower clients to express opinions and believe their opinions matter Not random

All suggestions receive equal treatment, whether large or small, commonor rare, etc.

Utility Clients give incomplete informationThe clinic quality committees do not have direct access to the box

Feasibility/ Easy to implement Many clients never notice the box; others do not know its purposeMinimal cost Some clients do not like to write or are illiterate

Clients often feel it is a waste of timeClients may prefer to give a face-to-face complaint to increase the likelihoodthat someone will take it seriouslyRequires a certain level of client empowermentClients do not know who will receive the informationClinics lack private space for completing forms

Direct Observation of Community Meetings

Utility Indicates the level of commitment of the community, which was May not be interactive enough for community representatives to raise clienthelpful to the research team complaints (communication at Max Salud was one-way: from clinicSome information about the community’s opinion of clinic services personnel to clients)

The meeting may not air issues suitable for QI

Feasibility/ Knowing cultural context facilitates data interpretation and analysis Data collectors had to rely on social workers to inform them of planned eventsParticipants were often fewFew male or adolescent participants

V. Findings: Analysis ofClient Feedback SystemAs the findings on client satisfactionshow, the combined data collectionmethods gave rich insights intoclients’ perspectives on servicequality. However, to implement all the

methods required significant re-sources. A primary purpose of thestudy was to assess different datacollection methods according to threecriteria: relative validity, utility, andfeasibility/cost. The assessment wasbased on interviews with datacollectors and clients who had

participated in the study, focus groupdiscussions with Max Salud qualitycommittees, and the research team’sobservation of the data collectionprocess. This section compares thedata collection methods in terms ofthe criteria.

Cost

Cost

Cost


A. Advantages anddisadvantages of the datacollection methodsUnderstanding the advantages anddisadvantages of each method withrespect to validity, utility, and feasibil-ity/cost allows managers to weighthese considerations and select thebest methods relative to individualpriorities and resource constraints.

■ Validity is the degree to which thedata seems to accurately capturewhat it is supposed to capture;validity is of primary importancebecause it indicates the extent towhich the objectives of the datacollection will be achieved

■ Utility refers to how useful theinformation for quality improvementactivities; utility is the second mostimportant criteria because itindicates the extent to which theresults of the data collection will beused

■ Feasibility/cost refers to how easyor difficult and how costly it is toobtain participants, apply the tools,analyze findings, present results,and routinely use the data collec-tion method

Table 3 summarizes the strengths andweaknesses of each data collectionmethod used in the client feedbacksystem, according to these criteria.

VI. Findings: Use ofClient Feedback atMax Salud

Feedback meetings with the qualitycommittees sought to make the clientsatisfaction data “actionable ratherthan dust-collecting” (Shelton 2000).The committees reacted very posi-tively to the results: several membersexpressed appreciation for the effort

and the richness of the data collected.They reported that the data gave thema concrete way to evaluate theirservices. The Balta Clinic director said,“Without this information, we wouldnever know these problems.” Thefeedback fostered the use of data inQI activities. The examples belowillustrate some of the ways that MaxSalud staff used client satisfactiondata to improve services.

A. Quality committee of themanagement support unitThe MSU quality committee found theresults useful because they revealedthe clinics’ advantages and disadvan-tages compared to other public andprivate facilities. This informationhelped Max Salud address weak-nesses and market its strengths. Theresults also helped the MSU qualitycommittee correct misperceptions.For example, in response to clients’expressions that interns were provid-ing clinic services, managers urgedstaff to tell clients they are fully trained,certified professionals. Balta Clinicmanagers also addressed clients’complaints about emergency servicesby increasing supervision of nightshifts to ensure that emergencyservices were provided consistentlyand carefully.

Throughout the data collectionphases, the research team immedi-ately reported any serious clientcomplaints to the Medical ServicesDirector. In some cases, the MedicalServices Director and Project Directormet with staff to address complaints.For example, managers coached agynecologist to improve his interper-sonal skills. In another situation,managers responded to a client’sconcern that a provider had been rudeby meeting with the provider to warnhim that he would be disciplined ifcomplaints continued. This situation

also caused managers to discuss theneed for a guideline calling for thepresence of a third party during clinicalexaminations. The guideline wouldprotect clients from inappropriatetreatment and providers from falseaccusations.

An unexpected benefit of collectingclient data was that it gave managersconcrete results to show to donors.Much of the data confirmed manag-ers’ sense that clients appreciated thefriendliness of personnel and promptservice more than other features ofMax Salud compared to other healthservices. Although the feedbackmeetings concentrated on clientdissatisfaction, in general Max Saludwas highly rated by almost all clients.Max Salud was able to present thisinformation to donors to show themtheir successes.

B. Balta quality committeeThe Balta quality committee foundclients’ perspectives useful in evaluat-ing the quality of services. The resultsconfirmed the need to expand theclinic in order to address complaintsabout size and congestion in thewaiting area during peak hours. Plansto expand the facility were alreadyunderway. The results also gave thecommittee members ideas on how toredesign the clinic layout. Based onclient feedback, the committeeconsidered how to address theclients’ need for a child care facility,more privacy in the consultation room,and more benches in the waiting area.

Staff made several key improvementsin admissions based on the clientsatisfaction data. Complaints ofteninvolved confusion about the hours ofoperation, misperceptions of certainadministrative processes, andfrustration with inefficient client flow.Balta staff began making the hours of


operations clear to clients, especiallyfor medical specialists who were part-time. They also began explaining thatclients with appointments took priorityover walk-in clients. This informationwould encourage clients to makeappointments and prevent feelings ofpreferential treatment.

The clinic director presented thefindings to the entire staff, whichallowed a self-assessment for staffmembers by sensitizing them to clientconcerns. Although the findings werepresented without the names ofclients or providers, the staff memberscould sometimes recognize them-selves in clients’ comments andrealized how clients interpreted theiractions. The expectation is that havingthe data would help staff to stepoutside their own viewpoints to seeinteractions from clients’ perspectivesand that this experience wouldchange staff attitudes and behaviors.

C. Urrunaga qualitycommitteeThe Urrunaga quality committee hadthe unique opportunity of using theclient satisfaction data in a trainingcourse on QI methods.4 The membersof the Urrunaga quality committeedecided to focus on client satisfactionduring the training because they wereconcerned about low utilization rates.They formed a team to analyze theclient satisfaction data, while learningand practicing the use of QI tools(flowcharts, fishbone diagrams,brainstorming lists, decision matrixes,tables, and graphs). The team devel-oped indicators and output standards,and compared their standards to theexit interview results from the first waveof data collection (see Table 4).

Table 4Standards Developed by the Urrunaga Team versus

Client Exit Interview Results

Indicator

Standard Result(Percentage) (Percentage)

Good ImageClients who say that Max Salud is the same as or better than other health centers 95 94

Clients who say they would return to the clinic 95 100

Clients who say they would recommend services to family/friends 90 100

Access to ServicesClients who say that prices are either average or low 95 97

Clients who do not experience any difficulties in visiting the clinic 80 58

Clients who say the hours of operation are not a difficulty 80 88

Clients who wait less than half an hour 90 66

Clients who say that waiting time is either average or short 90 72

Personnel PerformanceClients who leave without any unanswered questions 100 95

Clients who say that the provider treated them kindly 100 100

Clients who say that the provider greeted them 100 92

Clients who say that the provider respected their privacy 100 100

Client PreferencesClients who say there is nothing about the clinic that they dislike 80 62

Table 5

Decision Matrix to Select Best Solution Strategy

Solution Strategy Importance Feasibility Impact Cost Score

Implement scannable member cards and 5 4 5 3 17numbered tickets classified by colors accordingto services, and increase the participation ofhealth promoters as facilitators and hostesses.

Move free in-service supplies to consultation 5 5 4 5 19rooms, give the pharmacy an updated price list,train paramedical personnel to perform simplesutures, and provide emergency services only forcases that merit emergency care.

Avoid repetitive steps; redesign prescription, 4 5 4 5 18laboratory, and X-ray forms, and print them inlegible block letters.

4 For an account of the Urrunaga team’s problem-solving process, see Santillán 2000.


■ Update data collection toolsperiodically. Client satisfaction andperceptions of quality care arehighly dynamic. Cost may be asource of dissatisfaction at onepoint and later disappear as qualityimproves. Client expectationschange over time, which affectspeople’s level of satisfaction withexisting facilities. Some sources ofdissatisfaction are relatively simpleto solve (e.g., more benches in thewaiting area) and, once addressed,will not appear in subsequentassessments of satisfaction.Because of this dynamic environ-ment, programs should re-examinethe results of past data collectionactivities and update data collec-tion tools. Some issues may beexpanded and others droppedfrom the questionnaire anddiscussion guides.

■ Keep questions open-ended. Whilemore structured methods may beeasier to administer and analyze,data collection methods should alsopermit clients to express issues thatresearchers may not anticipate, anddata collection tools should bedesigned to accommodate thispossibility. Sensitive issues, such asinterpersonal relations and commu-nication, may require extra probingby data collectors, since clients mayhesitate to discuss them fully at first.

■ Use follow-up visits with dissatis-fied clients as part of qualityimprovement activities. Follow-upvisits were unplanned at the start ofthe study and were initiated toimprove staff’s understanding ofproblems or clients’ suggestionsfor improvements. In some cases,following up a complaint was part

Figure 2

Reducing Client Waiting Times at Urrunaga Clinic

100%

0%

20%

40%

60%

80%

Clients who waited30 minutes or less

Clients who reportwaiting was eitheraverage or short

99%

56%

80%

72%

Client satisfaction standard, 90% Results:September 1998

Results:July 2000

The team identified long waiting timesas the key cause of dissatisfactionand developed solutions to address it.The team used a flowchart to analyzeclient flow and brainstormed todevelop an exhaustive list of solutions.Next, they combined solutions that fitwell together from a systems view todevelop three sets of related solutions,or “solution strategies.” Staff voted fortwo strategies of the basis of eachone’s importance, feasibility, impact,and cost. Table 5 shows the results ofthe voting and indicates that thesecond strategy was selected.

After the study, Max Salud continued tocollect client satisfaction data. A rapidassessment in July 2000 showed aremarkable improvement in waitingtimes: clients who waited half an houror less rose from 56 percent to 80percent. Only one out of the 89interviewed clients responded that theirwaiting time was too long, demonstrat-ing an improvement from 72 percent to99 percent for clients who reportedthat their waiting time was eitherregular or short. (These results are

shown in Figure 2.) Thus, the Urrunagaquality committee was able to effec-tively use the client satisfaction dataduring team-based problem solving toimprove client flow and reduce waitingtimes.

VII. Lessons Learned:Improving ClientFeedback

Based on the analysis of the clientfeedback system, the followinglessons learned should be consideredwhen collecting and using clientsatisfaction data to improve the datacollection, integrate findings, andoptimize the use of data for qualityimprovement.

■ Give timely feedback. No matterwhat collection method is used togather client satisfaction data, theinformation should be processedquickly and presented to healthmanagers and/or quality teams.Short turnaround ensures rapid

responses to problems. Repeateduse of the data collection methodsresulted in shorter timeframes forboth collection and feedback.


of the solution, because the visitshowed Max Salud’s concern forits clients.

■ Carefully consider using focusgroups. Focus group discussionsare useful, but not very feasible.Organizing one is more labor-intensive than other methods, andconsiderable time is needed fordata transcriptions and analysis.Focus group discussions also takeconsiderable skill to administer andinterpret—a skill that few facilitieshave among their staff. A well-directed focus group can create asocial dynamic where clients shareexperiences with each other. To dothis effectively requires a skilledfacilitator.

■ Keep in mind that discontinuedclients may not be dissatisfied.Household follow-up interviews areuseful for reaching discontinuedclients. It is important, however, tounderstand that many clients ceaseattending health services forreasons not related to the quality ofservices. Some clients may havemoved or may not have neededadditional healthcare.

■ Recognize that a suggestion boxhas limited value as a clientfeedback mechanism. The

suggestion box is the least costlyof all the data collection meth-ods, and it gives clients animmediately visible opportunity toexpress their opinions. However,if health managers do not use itto improve quality of services, itbecomes solely a clinic fixturethat neither personnel nor clientstake very seriously.

■ Use community meeting observa-tion in conjunction with otherethnographic techniques. Theobservation of community meet-ings could have yielded moreuseful information in this study if ithad been used in conjunction withother techniques, such as observa-tion in the communities. Thecontextual knowledge that obser-vation of community actionprovides may be better gathered inmeetings other than those orga-nized by the clinic (e.g., mothers’clubs, worker unions, etc.) and atcommunity events.

■ Give quality committees time,training, and follow-up to maximizethe use of client satisfaction data.Quality committees need sufficienttime to thoroughly analyze andrespond to the client satisfactiondata and to develop action plans.By using the data in their QI

training, the Urrunaga qualitycommittee learned how toapproach the data systematically.

■ Use both quantitative and qualita-tive data. Both are useful and donot seem to be duplicative.Qualitative data is importantbecause it often captures specificclient complaints that managementcan respond to promptly, althoughexploring problems in interpersonalrelations requires time and sensitiv-ity. QI tools for using qualitativedata would optimize the use of thisrich data, ensure rapid responsesto specific complaints, andinstigate process improvements.

■ Explore the use of health promot-ers to collect client complaints.Health promoters are a crucial linkto the communities, although theyare traditionally used for clinic-to-client community mobilization,rather than client-to-clinic feed-back. Health promoters couldgather client complaints from thecommunity.

VIII. Conclusion

The Max Salud client feedback systemgenerated important information aboutthe quality of services from the clients’perspectives. Quality committees weresuccessfully able to use this informa-tion for a variety of quality improve-ment activities, ranging from rapidmanagement responses to improve-ments in organizational processes.Knowing the advantages and disad-vantages of the various methodsshould assist health managers inselecting appropriate methods tocollect clients’ perceptions andmaximize the use of that informationby quality committees. It is importantto weigh any potential trade-offsamong the validity, utility, and feasibil-ity/cost of different methods. Where


resources are scarce, health manag-ers are advised to choose a pack-age of methods that would besustained over time, because a one-time data collection has little value.Ultimately, no client feedback systemwill succeed without a dedication toproviding high quality healthcare thattruly places clients at the center ofservice delivery systems.

References

Aharony, L., and S. Strasser. 1993.Patient satisfaction: What we knowand what we still need to explore.Medical Care Review 50(1):49-79.

Arnetz, J., and B. Arnetz. 1996. Thedevelopment and application of apatient satisfaction measurementsystem for hospital-wide qualityimprovement. International Journal forQuality in Health Care 8(6):555-66.

Bartlett, E., M. Grayson, R. Barker, D.Levine, A. Golden, and S. Libber.1984. The effects of physiciancommunication skills on patientsatisfaction, recall and adherence.Journal of Chronic Disease37(9,10):755-64.

Bernard, R. 1995. Research Methodsin Anthropology. 2nd ed. London:AltaMira Press.

Brown, L., L.M. Franco, N. Rafeh, andT. Hatzell. Undated. Quality assuranceof health care in developing countries,2nd ed. Bethesda, MD: QualityAssurance Project.

Cleary, P., and B. McNeil. 1988.Patient satisfaction as an indicatorof quality care. Inquiry 25:25-36.

DiMatteo, M., and D. DiNicola. 1983.Achieving Patient Compliance: ThePsychology of the MedicalPractitioner’s Role. New York:Pergamon Press.

Donabedian, A. 1988. The quality ofcare: How can it be assessed?Journal of the American MedicalAssociation 260(12):1743-48.

Lochman, J.E. 1983. Factors relatedto patients’ satisfaction with theirmedical care. Journal of CommunityHealth 9(2):91-109.

Ovretveit, J. 1999. A team qualityimprovement sequence for complexproblems. Quality in Health Care8:239-46.

Pascoe, G. 1983. Patient satisfactionin primary healthcare: A literaturereview and analysis. Evaluation andProgram Planning 6(3,4):185-210.

Press, I., R. Ganey, and M. Malone.1990. Satisfied patients can spellfinancial well-being. Healthcare FinancialManagement 45(2):34-36, 38, 42.

Santillán, D. 2000. Using clientsatisfaction data for quality improve-ment of health services in Peru. CaseStudy. Bethesda, MD: Published forthe U.W. Agency for InternationalDevelopment by the Quality Assur-ance Project.

Scott, A., and R. Smith. 1994.Keeping the customer satisfied: Issuesin the interpretation and use of patientsatisfaction surveys. InternationalJournal for Quality in Health Care6(4):353-59.

Shelton, P. 2000. Measuring andImproving Patient Satisfaction. MD:Aspen Publishers.

Tavrow, P. 1997. Client satisfaction offamily planning services in developingcountries. Ph.D. Dissertation. Univer-sity of Michigan.

Thompson, A., and R. Suñol. 1995.Expectations as determinants ofpatient satisfaction: Concepts, theoryand evidence. International Journal forQuality in Health Care 7(2):127-41.

Wartman, S. 1983. Patient under-standing and satisfaction as predictorsof compliance. Medical Care21(9):886-91.

Williams, T., J. Schutt-Ainé, and Y.Cuca. 2000. Measuring familyplanning service quality through clientsatisfaction exit interviews. Interna-tional Family Planning Perspectives26(2):63-71.

Date post:	14-Feb-2017
Category:	Documents
Upload:	truongcong
View:	222 times
Download:	0 times

Implementing a Client Feedback System to Improve the Quality of ...

Documents