Advertisement

External Validation of Natural Language Processing Algorithms to Extract Common Data Elements in THA Operative Notes

Published:October 21, 2022DOI:https://doi.org/10.1016/j.arth.2022.10.031

      Abstract

      Background

      Natural language processing (NLP) systems are distinctive in their ability to extract critical information from raw text in electronic health records (EHR). We previously developed three algorithms for total hip arthroplasty (THA) operative notes with rules aimed at capturing (1) operative approach, (2) fixation method, and (3) bearing surface using inputs from a single institution. The purpose of this study was to externally validate and improve these algorithms as a prerequisite for broader adoption in automated registry data curation.

      Methods

      The previous NLP algorithms developed at Mayo Clinic were deployed and refined on EHRs from OrthoCarolina, evaluating 39 randomly selected primary THA operative reports from 2018 to 2021. Operative reports were available only in PDF format, requiring conversion to “readable” text with Adobe software. Accuracy statistics were calculated against manual chart review.

      Results

      The operative approach, fixation technique, and bearing surface algorithms all demonstrated perfect accuracy of 100%. By comparison, validated performance at the developing center yielded an accuracy of 99.2% for operative approach, 90.7% for fixation technique, and 95.8% for bearing surface.

      Conclusion

      NLP algorithms applied to data from an external center demonstrated excellent accuracy in delineating common elements in THA operative notes. Notably, the algorithms had no functional problems evaluating scanned PDFs that were converted to “readable” text by common software. Taken together, these findings provide promise for NLP applied to scanned PDFs as a source to develop large registries by reliably extracting data of interest from very large unstructured data sets in an expeditious and cost-effective manner.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to The Journal of Arthroplasty
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Melton G.B.
        • Hripcsak G.
        Automated detection of adverse events using natural language processing of discharge summaries.
        J Am Med Inform Assoc. 2005; 12: 448-457
        • Murff H.J.
        • FitzHenry F.
        • Matheny M.E.
        • Gentry N.
        • Kotter K.L.
        • Crimin K.
        • et al.
        Automated identification of postoperative complications within an electronic medical record using natural language processing.
        JAMA. 2011; 306: 848-855
        • Wyles C.C.
        • Tibbo M.E.
        • Fu S.
        • Wang Y.
        • Sohn S.
        • Kremers W.K.
        • et al.
        Use of natural language processing algorithms to identify common data elements in operative notes for total hip arthroplasty.
        J Bone Joint Surg Am. 2019; 101: 1931-1938
        • Sagheb E.
        • Ramazanian T.
        • Tafti A.P.
        • Fu S.
        • Kremers W.K.
        • Berry D.J.
        • et al.
        Use of natural language processing algorithms to identify common data elements in operative notes for knee arthroplasty.
        J Arthroplasty. 2021; 36: 922-926
        • Lee G.C.
        More data please! The evolution of orthopaedic research: commentary on an article by Cody C. Wyles, MD, et al.: “use of natural language processing algorithms to identify common data elements in operative notes for total hip arthroplasty”.
        J Bone Joint Surg Am. 2019; 101: e118
        • Fu S.
        • Wyles C.C.
        • Osmon D.R.
        • Carvour M.L.
        • Sagheb E.
        • Ramazanian T.
        • et al.
        Automated detection of periprosthetic Joint infections and data elements using natural language processing.
        J Arthroplasty. 2021; 36: 688-692
        • Tibbo M.E.
        • Wyles C.C.
        • Fu S.
        • Sohn S.
        • Lewallen D.G.
        • Berry D.J.
        • et al.
        Use of natural language processing tools to identify and classify periprosthetic femur fractures.
        J Arthroplasty. 2019; 34: 2216-2219