Skip to main content

Dr. Watson is Not a Meaningful User

IBM’s Dr. Watson of Jeopardy! fame has finally completed its residency and fellowships and, presumably to its creators’ utter delight, is now a practicing Oncologist. The prodigy “cognitive system” completed its training in less than a year at the illustrious Memorial Sloan-Kettering Cancer Center, and although only proficient in lung cancer right now, Dr. Watson’s career as an advisor to oncologists everywhere is off to a great start. A recently released video demonstration shows Dr. Watson in action, researching, evaluating and treating a 37 year old woman with newly diagnosed stage IV lung cancer in his advisory capacity to a hurried and pretty uninspiring human oncologist. Regardless of the slightly weird scenario, it is worth noting that in a fraction of a second Dr. Watson, scours 3,469 text books, 69 guidelines, 247,460 journal articles 106,054 other clinical documents and 61,540 clinical trials, and evaluates their contents against the patient’s EMR to identify need for further diagnostic tests and treatment options for this patient. Being an exceedingly helpful advisor, Dr. Watson quickly reads the entire EMR and uses his trained processing power to eliminate all the clutter in the EMR, presenting to the human doctor only information pertinent to this particular diagnosis. Ouch.

On the other side of town, ONC is busy apologizing for the sorry state of what it calls “interoperability”, blaming everything from the lack of standards to people’s inability to agree on a restricted set of vocabularies for the medical profession. According to the ONC philosophy of interoperability, only “computable” data can be exchanged or analyzed in a meaningful way. In other words, all medical professionals must learn to express themselves in a standardized way which computers can understand.  To that end we have ICD-9, ICD-10, SNOMED-CT, LOINC, RxNorm and all sorts of other terminologies and vocabularies aimed at restricting the English language to the limited computational abilities of available EMR software. How do you say “Mr. Smith is a pleasant 82 year old gentleman with a sad demeanor” in SNOMED? You don’t. You dispense with the pleasantries (pun intended) and diagnose Mr. Smith with depression. The Sapir-Whorf linguistic relativity hypothesis is by no means a settled subject, but if it contains any truth and vocabulary does affect cognition, then how will restricting clinical vocabulary affect the cognitive abilities of its users over time? We don’t know, and frankly I am not interested in finding out.

The folks at IBM took a different route. Paraphrasing Sir Francis Bacon, loosely quoted as, “If the mountain won't come to Muhammad then Muhammad must go to the mountain”, Dr. Watson’s creators must have decided at some point that if the doctor won’t come to the computer then the computer must go to the doctor. Instead of framing the problem by asking how we can change human communications to better enable the current generation of computers to “understand” humans, IBM began by figuring out how to change computers to better enable them to understand current forms of human communications. Thus, Dr. Watson learned to read books and articles and all sorts of “unstructured” information, because no matter how hard the powers to be are trying to fit the square peg of human language into the round hole of computer language, and tragically vice versa, most information generated by people is in their natural language and Dr. Watson was programmed to process natural language. So if Dr. Watson is able to “read” half a million pieces of text of various heft in a second or so, how long would it take for it to read an old fashioned paper chart, or an electronic rendition thereof? I am pretty sure that if you ask nicely, Dr. Watson would be happy to rearrange it for you in any way you choose, while pointing out the most pertinent parts to your current objective, highlighting discrepancies, missing and redundant information, all in a picosecond or less. And interestingly enough IBM developers thought it wise to take a generalized path to Watson’s education, instead of creating specialized Watsons each with linguistic abilities specific to a domain. Seems more human friendly that way…

The IBM Watson software line is not an EMR, but it can process and analyze information in an EMR. It is really an attempt at artificial intelligence consisting of a gigantic contextual search engine, coupled with lots of very sophisticated and self-generating algorithms to both analyze and inform the search itself. Watson doesn’t need to have the smoking status check box clicked in order to infer that the patient is or is not a smoker and doesn’t need to have a new standard defined before it can read a patient’s family history. True, Watson is pretty new software and folks have been tinkering with natural language processing and artificial intelligence for half a century without much success, but things are beginning to coalesce now and technology in the next decade will look very different. Is it really wise for our government to spend so much money and invest so much effort in building and enforcing the use of tools that are becoming obsolete faster than they are created? My hat is off to the VA and DoD who gave up on the strange and expensive idea of building their own EMR from scratch (better late than never). I think it’s high time that other governmental agencies got out of the EMR design business as well, because there are companies out there whose core competency is technology and who have large enough innovation budgets to build the next generation of health IT. Consider this: What if Dr. Watson had a few less educated siblings serving as medical secretaries, summarizing, abstracting and relaying information back and forth, on demand? All of a sudden the shape, form, functionality, standardization and all “meaningful” bells and whistles in an EMR are rendered irrelevant, and using Microsoft Word for typing or dictating your note is as good as using a “certified” EMR, or much better, because the context is so much clearer and so much more forthcoming.

Whether it can pass a Turing test or not yet, Dr. Watson is not a real doctor, and it will not be one in our lifetime. Dr. Watson has no free will and everything it knows is dictated by the corpus of knowledge made available to it by its creators or employers. There will be huge ethical and legal questions raised by software capable of supplanting human decision making processes, and software that can be centrally deployed and manipulated to this end. Even before that future arrives, it is worth noting that Dr. Watson is simultaneously employed in oncology clinics and by payers, and in my opinion, Dr. Watson has one button too many – the direct button to the insurance company, which will automatically approve payment for Dr. Watson’s top recommendations, but presumably not so much for other choices. Like all technologies, Watson embodies hope for the greater good along with great new perils for ordinary people. Leaving these philosophical questions aside for a moment, the only certain thing is that Dr. Watson is starting its brilliant new career by introducing a cure for one very painful disease that is reaching pandemic proportions amongst medical professionals: clicking boxes.

Comments

Post a Comment

Popular posts from this blog

The Holy EHR Wars

"In the same day the LORD made a covenant with Abram, saying, Unto thy seed have I given this land, from the river of Egypt unto the great river, the river Euphrates: " -- Genesis 15:18 Some four thousand years later, after numerous detours, major scope reductions, tragedies, compromises and constant upheaval, the Promised Land project is very much a messy work in progress. This is the nature of great promises, and in the here and now, we are contemplating yet another, albeit smaller, promise : “ “Hospitals, physicians and other health care providers are clearly taking advantage of recent incentives to embrace the promise  of technology [emphasis added] ,” said John R. Lumpkin, MD, MPH, senior vice president at the Robert Wood Johnson Foundation. ”  The Promise of Technology, as you may have guessed, was not made to us by the Lord Almighty, but instead it seems to be a product of spontaneous creation with mysterious origins. A few years ago, before the incentives mentioned b...
Post a Comment
Read more

The Big Birds of Health Care

For the first time in this election campaign Mitt Romney came up with a useful “zinger”. During the first Presidential debate Romney suggested that we should examine our various expenditures and ascertain if the item we spend taxpayers money on is “important enough to borrow money from China”. Unsurprisingly, the first thing that came to Romney’s mind was a public service providing small children with education on racial diversity and basic literacy skills. And although, we are not directly borrowing money from China to pay for things, it wouldn’t hurt to go through our expenses, including the many small and apparently insignificant ones, and see if there’s anything we can do without. It actually may be less painful to make a thousand additive little cuts than to locate one large silver bullet that is certain to cause commensurately large pain. Of course, such exercise would be fraught with controversy, since what may look frivolous to one party, could look worthwhile to another. But ...
Post a Comment
Read more

De Novo EMR Design Part II: The Quest for Information

The Edwin Smith papyrus ca. 1500 BCE In Part I of this series, we engaged in a design exercise for an imaginary software product that has no stated (or hidden) purpose other than to improve patient care. Following our initial definition of patient care, we formulated three general requirements and several constraints, none of which were specific enough to start building software tools from. The next step in our journey will break down each requirement into more specific tasks. What follows below will seem like an unnecessary and laborious statement of the obvious to some. However, I would submit that the careless bypassing of fundamental analysis is precisely what led us to where we are today, and even if we are forced to cut corners eventually, it is imperative that we define all corners prior to cutting them, instead of feigning shock and surprise after the fact. So without further ado, let’s start where we left off. System shall assist with gathering information from various sourc...
Post a Comment
Read more