Exploring Quality of HA standards: An Observational Study of HA Scoring Guidelines and Standards Compliance Evaluation
in Thailand
Ake-Chittra Sukkul1, Piyawan Limpanyalert* 1, Wiyawan Madsathan1, Thunyamas Sungnak1, Torranong Philalai1, Bussabong Wisetpholchai2
1The Healthcare Accreditation Institute (Public Organization), Nonthaburi, Thailand (www.ha.or.th), 2Thai Health Promotion Foundation, Bangkok
INTRODUCTION
•Since 2009, the Healthcare Accreditation Institute (HAI) has been a leading public organization in Thailand, committed to enhancing healthcare service quality and accrediting 953 hospitals by 2024. HAI collaborates with experts, member hospitals, and stakeholders to develop healthcare accreditation programs and tools, fostering a sense of co-ownership and understanding of HA standards.
•The HA standards, first published in 1996, have evolved through several editions in 2000, 2006, 2017, and 2021, with the 5th edition introduced by HAI alongside HA Scoring Guidelines, which became effective on October 1, 2022. These guidelines are crucial for assessing compliance and promoting quality development within healthcare organizations. Notably, the 5th edition of the HA Standards received international certification from ISQuaEEA in February 2022.
•This study seeks to evaluate the implementation of the HA Scoring Guidelines one year after transitioning from the guidelines of the HA Standard Edition 5th. The objective of this study is to document the phenomena observed during survey visits regarding the utilization of HA scoring guidelines for assessing adherence to HA standards and to provide insights into the validity and reliability issues associated with HA scoring guidelines through observational analysis.
•Challenges regarding the reliability of accreditation programs and interpretation of the HA Scoring Guidelines have emerged, necessitating an in-depth evaluation of their implementation.
AIM
The purpose of this study was to evaluate the effectiveness and quality of the Healthcare Accreditation Scoring Guidelines (HA Scoring Guideline) introduced with the 5th Edition of Hospital and Healthcare Standards in Thailand.
METHOD
- A phenomenon observation study characterized by a descriptive approach including surveys, focus group discussions, and phenomenological inquiry by non-participant observations during surveys and focus group seminars.
- Sampling Group: two healthcare organizations were selected, and surveyors were divided into two teams (Team A and Team B) to conduct comparative evaluations.
•Team A conducted authentic surveys.
•Team B shadowed Team A for comparative analysis.
-Data were collected on three primary dimensions: fact-finding, analysis, and scoring interpretation.
Study Steps:
- Reliability Test: Both surveyor teams reviewed the same hospital self-assessment forms and conducted surveys on the same hospital, scoring the operational level evaluation based on hospital standards.
- Validity and Sensitivity Testing: Focus group seminars were held to exchange opinions on scoring and testing results between the surveyor teams, experts, observers, and representatives of the Standards Development Subcommittee.
Finding indicated that HA Scoring varied significantly depending on the context of the hospital and timing of the evaluation. Both surveyor teams agreed on the consistency of fact-finding and analysis, particularly in recommending that hospitals comply with and strive to meet standards. However, discrepancies were noted in scoring interpretations, which were influenced by the abstract nature of the standards and the lack of clear explanations in the HA Scoring criteria.
Scoring Dilemmas:
1)Interpretation of Standards: Different interpretations arose due to the abstract nature of standards and varied understandings among surveyors, particularly in areas such as organizational management, hospital systems, and patient care processes.
2)Interpretation of HA Scoring: Ambiguities in the explanation of HA Scoring, such as unclear definitions of terms like "important issues" and "effective," led to inconsistencies in scoring.
RESULTS
Key Phenomena Result:
1) Relationships and Relationship Management: Observations revealed both formal and informal relationships between individual surveyors, potentially limiting their ability to express independent opinions about ratings. These pre-existing dynamics among team members may influence the scoring process and the level of agreement reached during evaluations.
2) Time Management: With a relatively limited time allocated for each visit to collect evidence and finalize scores, there were challenges in adequately exploring and documenting information necessary for scoring. The time pressure may have led to incomplete assessments or reliance on incomplete evidence, affecting the accuracy and reliability of the scores assigned.
3) Communication Management and Power Relations: Effective communication between surveyors and hospital staff is crucial for accurate assessments. Observations highlighted the presence of power dynamics between surveyors and the hospital staff, impacting the scoring process. Clear communication, particularly regarding the HA scoring criteria, could potentially mitigate these power imbalances and enhance the reliability of evaluations.
4) Impact of HA Accreditation Foundation Philosophy: The principle that "HA is an Education Process" influenced the surveyors' approach during visits, focusing on fostering learning and knowledge exchange. Transitioning from the previous version of HA scoring to a new model requiring deeper integration and analytical connection with HA standards posed challenges.
CONCLUSIONS
- Scoring for quality certification varies with hospital context and timing, hindering direct comparisons.
- Future efforts aim to enhance HA scoring's validity and reliability by adjusting methods, clarifying standards, and fostering supportive cultures. Despite ongoing HA standards reforms, challenges to accreditation reliability persist. Evaluating program validity and reliability during new standard implementations is crucial. Diverse surveyor perspectives could enhance credibility. Achieving HA scoring reliability remains a goal.
- Moving forward, efforts will focus on enhancing HA scoring's clarity, relevance, and analytical rigor for improved assessment confidence and reliability.
LESSONS AND LIMITATIONS
- The HA Scoring tools should be refined to increase their validity and reliability by reducing ambiguity in the standards and scoring criteria.
- Additional training for surveyors should be implemented to standardize the interpretation of abstract standards.
- The study process, including shadow surveys, should be utilized for developing surveyor skills in techniques, questioning, and assessment approaches.
- The study's reliability was limited by the equal experience levels of the surveyors and the similar quality development levels of the hospitals involved.
- The comparative assessment team had restricted access to information, as they were not allowed to conduct interviews or ask questions during the surveys.
ACKNOWLEDGEMENT
We would like to express our sincere appreciation and gratitude to hospitals, surveyors and the HA standard development committee.
REFERENCES
1.Greenfield D, Pawsey M, Naylor J, Braithwaite J. Researching the reliability of accreditation survey teams: lessons learnt when things went awry. Health Inf Manag. 2013;42(1):4-10. doi: 10.1177/183335831304200101.PMID: 23640917
2.Healthcare Accreditation Institute, Thailand. HA Standard, 5th edition. Available on https://www.ha.or.th (Thai version). For English version
CONTACT INFORMATION
The Healthcare Accreditation Institute
(Public Organization): www.ha.or.th
Ake-Chittra Sukkul: Ake-chittra@ha.or.th
Piyawan Limpanyalert: Piyawan@ha.or.th
Wiyawan Madsathan: Wiyawan@ha.or.th
