(Shot this in Alexandria, Virginia.)

Christmas lights, passers-by, wet snowflakes, a donation and a smile, and one hot saxophone version of my favorite Christmas tune: Carol of the Bells.

• 0:24 A tinkly riff, but a some false notes…
• 030: You realize he’s wearing thick gloves…
• 0:52 His sax begins to wail…
• 0:57 He frowns, off come the gloves…
• 1:07 Hmmm, better…
• 1:27 Adjust that mouthpiece…
• 1:55 Zoom in on his face…
• 2:00 A little improv…
• 2:10 Wow!
• 2:50 “Carol of the Bells!”…(yeah!)
• 2:58 “Merry Christmas all!”…

Merry Christmas all!

A.S. I publish this on Thanksgiving Day and give thanks for our American tradition of innovation. It is a unique product of personality (forebearers and forerunners), platform (laws and traditions,) and opportunity (Thanksgiving’s Land of Plenty).

“There is a dream dreamed by engineers and designers everywhere that they will someday be put in charge, and that their rigorous vision for the world will finally overcome the mediocrity around them once and for all. Resist this idea - the world does not work that way, and the dream of centralized control is only pleasant for the dreamer.” (Clay Shirky, An Open Letter to Jacob Nielsen)

As a graduate student in Industrial Engineering, studying aviation human factors and health systems engineering at the University of Illinois, I would have felt personally insulted by the above quote. Now that I’ve grown up, not so much.

Twelve years ago Clay Shirky wrote a remarkable open letter to a senior spokesman and advocate for Web usability, Jacob Nielsen, in defense of market-driven improvements of Web usability over centralized enforcement of usability standards. His remarks apply remarkably well to one side of the EMR / EHR usability certification debate. In the fine spirit of Shirky’s philosophies of open source and evolvable systems I’ve adapted his letter to the EMR/EHR usability standards debate.

(By the way, I don’t know if Nielsen still agrees with himself (heck, in the words of Marshall McLuhan, even “I don’t necessarily agree with everything I say.”). I certainly agree with Nielsen about big targets, less-is-more functionality, and screen flow that matches task flow.)

In some instances the Shirky quote is close to word for word, except the substitution of “EMR / EHR” for “website” or “EMR / EHR industry” for “the Web”, and in others I more liberally adapt the quote to current EMR / EHR industry circumstances. Quotes and paraphrases are blue and indented; my comments are black and unindented.

For maximum impact you might just read the indented blue material straight through, skipping over my comments until later.

Flame on!

An Open Letter about Enforcing EMR / EHR Usability Standards

Let me preface all of this by noting what we agree on:

• the EMR / EHR industry is host to some hideous dreck;
• things would be better for users if EMR / EHR designers made usability more of a priority;
• and there are some basics of interface usability that one violates at one’s peril.

So far, so good!

Where we disagree, however, is on both attitude and method - for you, every EMR / EHR is a piece of software first and foremost, and therefore in need of a uniform set of UI conventions, while for me, a EMR / EHR function is something only determined by its designers and users - function is as function does. I think it presumptuous to force a third party into that equation, no matter how much more “efficient” that would make things.

Depending on which side you’re on in the EMR / EHR usability standards debate, you may agree or disagree with this sentiment. However, it’s a close approximation of how some EMR / EHR developers feel.

What about patients, payers, public health agencies and clinical outcomes researchers? Don’t they have a stake? Yes they do: as users (and, in some cases, designers).

You despair of any systemic fix for poor EMR / EHR usability and so want some sort of enforcement mechanism for external usability standards….I believe that a market for quality is in fact the correct solution for creating steady improvements in EMR / EHR usability.

Let me quickly address the least interesting objection to your idea: it is unworkable. Your plan requires both centralization and force of a sort it is impossible to achieve. You say

“…to ensure interaction consistency across all EMRs / EHRs it will be necessary to promote a single set of design conventions.”

and

“…the main problem lies in getting EMR / EHR vendors to actually obey any usability rules.”

…I am relieved that there is no authority who can make EMR / EHR designers “obey” anything other than data interoperability….With the EMR / EHR use poised to go from 10-30 percent (depending on your definition of an EMR / EHR) to close to 100 percent in the next few years…the idea of enforcing usability rules will never get past the “thought experiment” stage.

The analogy breaks down a bit here. Government-enforced EMR/EHR usability standards may indeed get past the thought experiment stage.

…I want to address why I think enforced EMR / EHR conformity to EMR / EHR usability standards is wrong, even in theory. My objections break out into three rough categories: creating a market for EMR / EHR usability is better than centrally enforced EMR / EHR usability standards for reasons of

• efficiency,
• innovation, and
• morality.

EFFICIENCY

..The “Enforce EMR / EHR Usability Standards” Solution - redesign EMRs / EHRs not presently complying with a single set of usability conventions - takes care of 100% of the problem, while the “Create a Market for Usable EMRs / EHRs” Solution, let’s call it evolutionary progress for a highly usable EMRs / EHRs, well what could that possibly get you?

…a surprising amount, actually, if it’s properly arranged.

By ignoring the mass of EMRs / EHRs with just a few customers each and instead concentrating on making the popular EMRs / EHRs more usable and the usable EMRs / EHRs more popular, a market for quality is a more efficient way of improving EMR / EHR usability than trying to raise quality across the board without regard to user interest.

In other words, instead of raising average usability of all EMRs / EHRs, raise the usability of the most usable EMRs / EHRs through market-based innovation.

INNOVATION

A market for EMR / EHR usability is better for fostering innovation. Good tools let EMR /EHR designers do stupid things. This saves overhead on the design of the tools, since they only need to concern themselves with structural validity, and can avoid building to complex usability guidelines….

Consider the use of HTML headers and tables as layout tools. When these practices appeared, in 1994 and 1995 respectively, they infuriated partisans of the SGML ‘descriptive language’ camp who insisted that HTML documents should contain only semantic descriptions and remain absolutely mute about layout. This in turn led to white-hot flame fests about how HTML ’should’ and ’shouldn’t’ be used.

I recall these flame fest’s well. During the early 90’s I was part of a team of developers who created one of the first clinical browsers based on SGML. (A Tcl/Tk Based Graphical Interface to Medical and Administrative Information, presented at the 1994 18th Annual Symposia on Computer Applications in Medical Care)

It seems obvious from the hindsight of 1999, but it is worth repeating: Everyone who argued that HTML shouldn’t be used as a layout language was wrong. The narrowly correct answer, that SGML was designed as a semantic language, lost out to the need of designers to work visually, and they were able to override partisan notions of correctness to get there. The wrong answer from a standards point of view was nevertheless the right thing to do.

Remember the competition between networking standards and which one won? TCP-IP did, even though it was not deemed the most elegantly or correctly designed networking standard at the time. The market will do what the market will do.

Enforcing any set of rules limits the universe of possibilities, no matter how well intentioned or universal those rules seem. Rules which raise the average quality by limiting the worst excesses risk ruling out the most innovative experiments as well by insisting on a set of givens. Letting the market separate good from bad leaves the door open to these innovations.

Correctomundo!

MORALITY

This is the most serious objection to your suggestion that standards of EMR / EHR usability should be enforced. An EMR / EHR is an implicit contract between two and only two parties - designer and user. No one - not you, not Don Norman, not anyone, has any right to enter into that contract without being invited in, no matter how valuable you think your contribution might be.

Here is where the analogy between 1999’s consumer-facing website and todays EMR / EHR industry is, perhaps, but only perhaps, the weakest. What about patients, payors, public health agencies and clinical outcome researchers? Don’t they have a right to interfere in the contract between EMR / EHR user and EMR / EHR user because they are directly or indirectly affected, or in the name of public good? Perhaps.

On the other hand, these stakeholders are users with their own contracts with the EMR / EHR developer. I think one especially important role for the EMR / EHR usability engineer is to help create, and then enforce, the winningest set of user-designer contracts possible. However, while I think we can agree on this, it doesn’t shed much light on the government-needs-to-enforce-EMR-EHR usability-guidelines debate. Usability engineers on one side of the debate will favor one set of contracts while those on the other side of the debate will favor a different set.

By the way, I’ve met Donald Norman, and read and reread his books. I especially recommend Things that Make You Smart.

IN PRAISE OF EVOLVABLE SYSTEMS, REDUX

I believe that the EMR/EHR industry can be a market for quality -

• if switching costs can be lowered,
• word of mouth effects made large and swift, and
• redesign relatively painless.

If I design a usable EMR / EHR, I will get more repeat business than if I don’t. If my competitor launches a more usable EMR / EHR, it’s only a data export/data import away. No one who has seen the development of Barnes and Noble and Amazon or Travelocity and Expedia can doubt that competition helps keep sites focussed on improving usability. Nevertheless, as I am a man of action and not just a theorist, I am going to suggest a practical way to improve the workings of this market for usability - lets call it usable-emr-ehr.lycos.com.

This paragraph is, I think, key. The reason that some feel we need to “resort” to enforced EMR / EHR usability standards is because of a perceived market failure (not a uniform perception by the way). One byproduct of increased data interoperability (admittedly facilitated by government promulgated data standards) will make it easier and easier to switch between EMRs/EHRs. And as more and more EMRs / EHRs become web-based, there is less infrastructural lock-in too.

Word of mouth among physicians and along physician social networks is a powerful potential propellant (and deterrent) of EHR / EHR adoption. If switching costs can be lowered, harness this.

I am particularly interested in the phrase “redesign relatively painless,” because the Achilles heel of EMR / EHR usability has been frozen workflows that are expensive to change and which frustrate users. Most of the over one hundred posts on this blog are about technologies such as workflow management systems and business process management suites that could be used to address the problem of frozen EMR / EHR workflow.

So,

• reduce the cost of switching between EMRs / EHRs,
• rely on physician word of mouth and social media, and
• increase EMR / EHR flexibility of redesign (especially regarding workflow),

and the need for enforced EMR /EHR usability standards (and the unintended consequences) will diminish.

By the way, at CMU I took an artificial intelligence course from Michael “Fuzzy” Mauldin, Ph.D., before he founded Lycos. (Hi Fuzzy!)

The way to allocate resources efficiently in a market with many sellers (EMR/EHR vendors) and many buyers (users) is competition, not standards. Other things being equal, users will prefer a more usable EMR / EHR over its less usable competition. Meanwhile, EMR / EHR vendors prefer more EMR/EHR business to less, and more repeat EMR / EHR customers to fewer.

Econ 101 anyone? Actually, Production Possibility Frontier, but: “Production Possibility Frontier anyone?”

Imagine a search engine that weighted EMR / EHR usability in its rankings, where users knew that a good way to find a usable EMR / EHR was by checking the “Weight Results by EMR / EHR Usability” box and owners knew that an EMR / EHR could rise in the list by offering a good user experience. In this environment, the premium for good EMR / EHR UI would align the interests of buyers and sellers around increasing quality. There is no Commissar of EMR / EHR Design here, no Bureau of EMR / EHR Usability Standards, just an implicit and ongoing compact between users and designers that improvement will be rewarded.

When I first read this suggestion about usability-weighted search engine results, I was skeptical. But the more I thought about it the less skeptical I became.

I’ve read that Google sorts web pages using over 200 criteria and that they constantly tweak these criteria. I pay close attention to this sort of thing because I pay attention to where this blog ranks for certain phrases (“EMR + workflow”). I believe Google does in fact take into account website usability in its ranking system, that certain usability heuristics, those that can be search engine spider mechanized, do in fact give more usable websites a boost in their search ranking. At the very least, Google’s PageRank website link voting algorithm must reflect some aspects of website usability. All other things remaining equal, websites are more likely to link to more usable websites then less usable websites, and this should affect SERP (search engine result pages/position) via Google’s PageRank algorithm.

Of course, EMRs/EHRs are not websites, indexed in search engines, cross-linked so a PageRank-style Usability-Rank voting algorithm can rank them. Nonetheless, there is an interesting germ of an idea here. More later.

The same effect could be created in other ways - a Nielsen/Norman “Seal of Approval”, a “Usability” category at the various EMR / EHR awards ceremonies, a “Usable EMR / EHR Web Ring”. As anyone who has seen “Hamster Dance” or an emailed list of office jokes can tell you, the net is the most efficient medium the world has ever known for turning user preference into widespread awareness. Improving the market for quality simply harnesses that effect.

Ha! Remember web rings! The “Hamster Dance”? (wiki) I also like the idea of industry awards for EMR / EHR usability. (A self-serving disclaimer, I helped three medical practices win the first three consecutive HIMSS Davies Awards for ambulatory excellence. The submitted applications–accessible from previous link–include commentary about EMR / EHR usability.)

Web environments like usable-emr-ehr.lycos.com, with all parties maximizing preferences, will be more efficient and less innovation-dampening than the centralized control which would be necessary to enforce a single set of EMR / EHR usability standards. Furthermore, the virtues of such a decentralized system mirrors the virtues of the Internet itself rather than fighting them. I once did a usability analysis on an EMR / EHR which had fairly ugly but a good UI nevertheless. When I queried the EMR /EHR vendor about his design process, he said “I didn’t know anything when I started out, so I just built and EMR / EHR with an email link on every screen, and my customers would mail me suggestions.”

Open source EMRs / EHRs have not yet played much of a role in the EMR / EHR usability debate, or in usability in general (I love Ubuntu, but Windows 7 and Apple OSX provide better user experiences for the less technical minded). On the other hand, open source software has the advantage of a tight loop of interaction between user and programmer (à la EMR / EHR users emailing suggestions via links on EMR / EHR screens). Perhaps there is an opportunity here for open source EMRs / EHRs to exploit this potential advantage.

There is a dream dreamed by EMR / EHR usability engineers and EMR / EHR user experience designers everywhere that they will someday be put in charge, and that their rigorous vision for the EMR / EHR world will finally overcome the mediocrity around them once and for all. Resist this idea - the world does not work that way, and the dream of centralized control is only pleasant for the dreamer. The Internet’s ability to be adapted slowly, imperfectly, and in many conflicting directions all at once is precisely what makes it so powerful (would that EMRs and EHRs emulate this!), and the Web has taken those advantages and opened them up to people who don’t know source code from bar code by creating a simple interface design language (something EMRs and EHRs could use too!).

“[O]pened them up to people who don’t know source code from bar code by creating a simple interface design language”, in the long run, this is what needs to happen to the relationship between EMR / EHR users and EMR / EHR designers–they need to become one and the same. The key to this is user-customizable workflow. (But don’t get me started here, if you are interested you can read about it here, here, and here.) The EMR / EHR ecosystem needs to become more similar to the Web ecosystem. In which case the analogy between Web usability and EMR / EHR usability becomes even stronger.

The obvious short term effect of this has been the creation of an ocean of bad EMR / EHR design, but the long term effects will be different - over time bad EMR / EHRs die and good EMRs / EHRs get better, so while those short-term advantages seem tempting, we would do well to remember that there is rarely any profit in betting against the power of the marketplace in the long haul.

I couldn’t have said it better myself! (Wait a minute, I just did! In an open source, evolvable, attributed sort of way.)

EPILOGUE

The biggest difference between Shirky’s 1999 websites and today’s EMRs / EHRs is potential impact of EMR / EHR-induced medical error on patient safety. If someone can’t find their way around Amazon or Orbitz, so what. If a physician cannot find his or her way around an EMR / EHR, a critical piece of missing patient information might result in disaster for that patient. Narrowly focusing EMR / EHR usability on patient safety concerns–publishing suggested formats, an EMR / EHR incident database, educating EMR / EHR users about EMR / EHR usability–are good ideas.

However, keep in mind that standards always reduce innovation…somewhere…in hope of increasing innovation elsewhere. Draconian enforcement of EMR / EHR usability standards may increase EMR / EHR usability in the short run, but decrease the EMR / EHR usability (including patient safety) in the long run.

P.S. Follow me on Twitter at @c_wb.

In spite of yesterday’s prediction, here are some photos:

Several days before, looking east…

…then the autumn equinox, September 23 evening, the sun reflecting back from glass-surfaced building many blocks away…

…same reflection seen parallel with sidewalk…

…then looking west toward the Friendship Arch from several blocks away…

…Cheers!

P.S. Follow me on Twitter at @c_wb.

The Office of the National Coordinator for Health Information Technology (ONC) recently published the Federal Health Information Technology Strategic Plan for 2011 – 2015 (announcement, pdf).

The word “usability” is used twenty-five times.

Not going to read the full 80 pages (shame on you!) but want to stay up to date about government plans for improving EMR EHR usability? You came to the right blog post. I’ve abstracted each use of “usability” below. Emphases in bold are my own.

P.S. I like the HealthIT.gov’s tagline: Putting the “I” in Health IT℠. So much so, I’m going to use the parallel-themed tagline, Putting the “U” in EHR Usability℠, for a new website I working on. “U” refers to “you” the “user”. Stay tuned!

From the strategic plan:

pp 12-13

Strategy I.A.9: Encourage and facilitate improved usability of EHR technology.

The government is collaborating with industry and researchers to improve the usability of EHRs. The usability of EHRs is considered a key barrier to adopting health IT and achieving meaningful use. NIST is conducting ongoing research and advancing the development of standards and test methods that can be used to evaluate and improve the usability of EHRs. It has released a Common Industry Format (CIF), a standard for developers to report usability test findings and demonstrate evidence of usability in their products in a format that allows for independent evaluation of a single product and comparison across multiple products. NIST is also developing guidance and tools for RECs and professional societies on available tools and resources to incorporate concepts of usability in selecting and implementing EHR systems. The Food and Drug Administration (FDA), in collaboration with NIST and the Agency for Healthcare Research and Quality (AHRQ), will develop best practices to address systematic evaluation of usability with regard to patient safety to ultimately improve patient care. AHRQ is developing toolkits that medical practices can use to assess the usability of EHR systems and assess the redesign workflow. In addition, AHRQ is conducting research and convening industry workgroups that provide perspectives on what constitutes usability and how to systematically improve the usability of EHRs.

ONC will explore ways to improve the ability of providers to select or change EHR products by improving data portability. Reducing the cost associated with switching products while increasing data fluidity and choice can help drive market competition to improve the usability of EHR products.

ONC has directed one of its four Strategic Health IT Advanced Research Projects (SHARP) (see Strategy V.B.2) to further EHR usability through the identification and development of better cognitive and user-centered design. In addition, ONC is working with private sector groups to encourage the collection of usability information and its dissemination to vendors and consumers through mechanisms they can trust.

p 34

OBJECTIVE C Improve safety and effectiveness of health IT

Strategy III.C.1: Provide implementation and best practice tools for the effective use of health IT.

AHRQ’s Health IT Portfolio supports health services research grants and contracts that create new knowledge, synthesize and disseminate best evidence and provide tools for implementation addressing health IT’s impact on the quality of health care. Current initiatives address clinical decision support, patient safety, patient centered care, quality measurement, and usability and workflow issues. In addition, ONC is actively working to make resources available to providers that allow them to maximize the value of using health IT by avoiding common challenges and legal issues associated with adoption, implementation, and use of EHRs and other health IT. Professional societies, licensing boards, and continuing education programs are developing best practice resources related to issues such as workflow redesign, the need for ongoing maintenance and upgrades, and legal concerns related to vendor contract clauses. There are important legal issues that providers should be aware of when entering into agreements with EHR and other health IT vendors. ONC will work to equip providers with information and help address potential barriers they may face in achieving meaningful use, including improved usability of EHR technology (see Strategy I.A.9). ONC and RECs will ensure that appropriate best practice resources about these issues are distributed to the providers that need them.

p 41

Accommodate the range of user capabilities, languages and access considerations, including effective strategies for ensuring accessibility and usability of electronic health information for people with disabilities and meaningful access to such information for individuals with limited English proficiency

p 47

Accommodate the range of user capabilities, languages and access considerations, including effective strategies for ensuring accessibility and usability of electronic health information for people with disabilities and meaningful access to such information for individuals with limited English proficiency

p 48

Strategy V.B.2: Make targeted investments in health IT research. The federal government is committed to investing directly in health IT research and development in areas that hold great promise for improving the health of individuals and populations. NIH and ARHQ, in particular, are funding dozens of research projects related to the development of health IT. Through the HITECH Act, ONC established the Strategic Health IT Advanced Research Projects (SHARP) Program, a four-year program funding research in health IT security, patient-centered cognitive support, health care applications and network platform architectures, and secondary use of EHR data. The progress made by grantees will assist in developing best practices which can be applied nationwide, possibly through meaningful use requirements. AHRQ sponsors projects focused on best practices and integration of health IT into the practice of medicine. Focus topics include usability of EHRs, clinical decision support, consumer health IT, health information exchanges, and telehealth.

p 51

Appendix B: Programs, Initiatives, and Federal Engagement

NIST collaborates with HHS/ONC in realizing the health IT goals of the Administration and Congress. This relationship allows ONC to draw upon NIST expertise in applying IT to health care through standards, conformance measurement, prototype implementation, security, and usability, and in consulting on the Nationwide Health Information Network, standards, and certification processes.

p 52

NIST has funded a grant on the “Relationship Between Health IT Usability and Patient Safety: A Human Factors Engineering Framework for Action.” This grant will develop a framework explaining how the multiple facets of usability may be linked to different aspects of patient safety.

AHRQ’s Health IT Portfolio has supported health services research grants and contracts that create new knowledge, synthesize and disseminate best evidence and provide tools for implementation addressing health IT’s impact on the quality of health care since 2004. Current initiatives address clinical decision support, patient safety, patient centered care, quality measurement, and usability and workflow issues.

p 60-61

Testing the usability and feasibility of smartphone-based applications and patient links to clinical services, including, but not limited to a recovery tool called Addiction Comprehensive Health Enhancement Support System (A-Chess) – an online peer support group and clinical counselors, a GPS feature that sends an alert when the user gets near an area of previous drug or alcohol activity, real-time video counseling, and a “panic button” that allows the user to place an immediate call for help with cravings or triggers.

p 70

Appendix F: Goals, Objectives, and Strategies

Goal I: Achieve Adoption and Information Exchange through Meaningful Use of Health IT

Objectives

Strategies

I.A. Accelerate adoption of electronic health records

I.A.9. Encourage and facilitate improved usability of EHR technology.

In previous posts I’ve argued that most traditional EMRs EHRs with desktop interfaces (monitor, keyboard, and mouse) are wrongheaded and misguided. Instead of mimicking Microsoft’s explicit and implicit user guidelines, resulting in EMRs EHRs that resemble Microsoft Office, they should copy smartphones and tablets.

I’ve written that EMRs EHRs need to

• have larger buttons (or “targets” in aviation human factors vernacular)
• have fewer buttons per screen
• have more screens reflecting task-specific workflows

and to do this EMRs EHRs need to become more context-ware/process-aware.

In short,

• EMRs EHRs need to become more intuitable; what most non-usability professionals mean when they call for more “intuitive” EMRs EHRs.
• EMRs EHRs need to become more intuitive; meaning the EMR EHR more intelligently and actively “intuits” user goals and facilitates task workflows.

Neilsen does not appear to have publicly weighed in on EMR EHR lack of usability. But he has commented about smartphone usability experience. His comments are right on, and just as relevant to EMR and EHR usability.

Neilsen says:

“What are the biggest mistakes companies make when building their sites and apps?

1. Making it difficult to touch and manipulate. As Nielsen explains, your eyes are more agile than your fingers. You see that link among but your fingers can’t select it, so Nielsen recommends using larger touch targets.
2. Trying to do too much. Those responsible for the mobile UE must be ruthless when fighting internal political battles. Every department wants their content front and center, but when everything is prominent, nothing is prominent, so eliminate the nice-to-have.

Nielsen argues for fewer commands and a few basics on first screen. Use progressive disclosure to build the experience, rather than try to put every option up front. More screens are better if each is simple and focused….

Without context, Nielsen reports, comprehension is degraded. While our brains are great for long-term storage, they fail in the short term. For this reason, people will not use mobile for research or comparing large amounts of information. Don’t require a user to remember things from screen to screen.”

So, consistent or inconsistent with what I’ve been posting?

I say: Consistent!

Case closed! Neilsen would agree with me. (I’m so smart!)

Wordle based on sixteen short quotes
from the awarding winning application

I regularly scour the Web looking for evidence of convergence between the worlds of EMR/EHR and workflow management systems/business process management (check out the EHR.BZ REPORT on EMR / EHR Workflow, Usability and Productivity). To quote myself:

“The next step in the evolution of ambulatory EMRs is squarely at the intersection between two great software industries: electronic health record systems and workflow management/business process management systems. The hybrid EMR workflow systems that result will be more usable and more systematically optimizable than traditional EMRs with respect to user satisfaction, clinical performance, patient satisfaction, and practice profitability.” (A White Paper About EMR Workflow, Usability, and Productivity in Pediatric and Primary Care)

Eight years before that I co-authored a paper about the first EMR / EHR workflow management system, about which I wrote:

“Developments that are beginning to affect collective thinking of the HIT industry today:

• Workflow management systems (business process management today)
• Computer-supported collaborative work (groupware and workflow systems today)
• Componentized EMR architecture (software modules/plugins and service-oriented architecture today)”

(Ten Years Ago, Dallas HIMSS: Landmark Presentation on Modular Pediatric EMR Workflow Groupware)

It’s too soon to write “I told you so!” but that delicious moment draws ever closer. Health information technology and business process management industries increasingly overlap and interact, giving rise to much needed hybrid vigor.

Below are quotes from the case study. If you follow healthcare discussions about EMR / EHR usability, interoperability (especially workflow, or pragmatic, interoperability), clinical intelligence, population health management, clinical groupware, collaboration, coordination, continuity of care, they will surely pique your interest. Skim the bold emphases I added and think (”Just think!”) about their relevance to solving many of the problems plaguing today’s generation of EMRs and EHRs.

Mind the Gap! (those strategically placed three-dot thingies)

Teamwork, collaboration and coordination are key aspects of the patient-centric approach taken by modern healthcare….The work described includes tracking care teams and individual team members dynamically as the patient progresses along the dynamic care pathway. It proposes the integration of a Business Process Management (BPM) system into the HISs as it will better support both the individual work of health and care practitioners as well as improve support for team communication, and care coordination throughout the patient’s care….The proof of concept prototype showed that the application of Workflow Technology in the healthcare domain is a very promising development. It can be used to evolve the functionalities of existing HIS, so that they can be used to support implementation of ICP services and associated treatment flow for a patient. It is believed that these functionalities are important as they result in safer more effective and efficient care and treatment. Functionalities of the proposed system include: providing a pro-active system, routing and information filtering….The strength in the BPM systems is in its ability to invoke existing systems at any stage during the process flow in a way which overcomes the system heterogeneity challenge….BPMs can be adjusted to enforce a specific sequence and\or enable an extremely flexible order of processes….At a business level, the system will facilitate multi-professional care team communication and therefore care coordination across the multiple healthcare organisations involved in the treatment of patients….The proof of concept prototype showed that the application of Workflow Technology in the healthcare domain is very promising….Functionalities of the proposed system include: providing a pro-active system, routing, task automation and information filtering. These functionalities are: Pro-active System: this is the primary advantage of using Workflow Technology in the healthcare domain. It is the difference between having a reactive or a proactive system. In the case of a reactive system, which most traditional HISs are, reactions are a response to requests made by users while proactive systems are capable of identifying the need to take an action and activities it activates. The workflow engine within the workflow management system can be coded to fetch triggers, understand which of the users or roles are affected by this trigger, how they are affected and finally, take appropriate actions to inform them. These pro-active functionalities can be used to execute many different actions, such as: alert, notify, refer, schedule and set timers….Routing: this helps the system determine the sequence of the processes and the consequences of any decision made. This is either by suggesting the next stage or automating a set of processes. While routing is a tool that WFMSs provide, routing is only made with a user’s approval or suggestion. This is done by providing a message to the user showing the alternative routes according to the ICPs’ logic and the user can approve any or simply skip it….Task automation: is performed when a number of tasks need to be processed as a set. In this context, it includes tasks that do not require user interaction….Information extraction and filtering: to ensure summary of important information is visible to healthcare professionals when viewing a patient’s records. This aims to facilitate tracking a patient and improving the decision-making process by making healthcare professionals aware of the development of the care process and therefore making better use of their time. This includes improved visibility of: treatment history, milestones, order and time, and acting healthcare professionals….At the system level, the HIS will become more pro-active and capable of performing the following actions: alert, notify, refer, schedule, and set timers. It also provides a flexible system that handles dynamic changes happening during a patient’s treatment. This includes routing the flow and performing automatic tasks. Moreover, the workflow system provides a tool to track patients and ensure continuity of the flow by filtering and extracting important information. The treatment information extracted includes: history, milestones, order and time and involved care team professionals….The coordination problem…need for a more proactive system that facilitates care coordination among care team members…Business Workflow: This is required to model the huge number of processes interacting in a healthcare system. Business workflows support human interaction with the system. This is necessary in healthcare systems, where different care team professionals interact with the system and support the decision making process and therefore the routing of the flow….Activities represent all the treatment and diagnosis options a patient can follow. These options should be modelled in the WFMS and form the main block of the system….A Process: this is the actual workflow map of the clinical guidelines…An Activity: the steps of the clinical guidelines….A Process object: this provides control or audit to the case according to its state….A case: this is the scenario in progress. Each case represents a patient’s treatment flow. For each patient, the treatment pathway is unique and is processed by considering a patient’s health condition and the available resources. The case hierarchy at run time usually shows treatment history, the progress, the state in each and the roles or users involved in different stages.

Again, congratulations…and thank you!

Last fall I presented a paper, co-written with Mark Copenhaver, at MedInfo2010 in Cape Town, South Africa.

Webster C. & Copenhaver, M. Process-aware EHR BPM Systems: Two Prototypes and a Conceptual Framework. In: Proceedings of the 13th World Congress on Medical Informatics, Studies in Health Technology and Informatics, Volume 160, 2010, pp 106-110. (indexed in MedLine)

You may have noted my photo travelogue at the time.

Process-aware health information system research and related industry undertakings have evolved since we built clinical intelligence and process mining prototypes in 2009 and thought how to bridge between healthcare IT and business process management. One of the goals of the blog is to draw readers, equally, from the realms of EMR/EHR/Health IT and BPM/Workflow/groupware. If you are from one of these industries but not the other, you’ll know some of the terminology, but not all. Hence a lengthy editorial preface plus a glossary of EMR / EHR workflow terminology as an addendum.

You can keep reading, skip to the abstract/slides/notes, or peruse the following outline and cut the (slide) deck wherever you like.

• Short “Editorials” on EMR, EHR, BPM, BI, CI, CEP, Productivity and Usability
• Process-aware EHR BPM Systems: Two Prototypes and a Conceptual Framework (slides & notes)
• EHR Productivity = Information Value / Information Cost
• Closed-Loop Optimization can Systematically Improve EHR Productivity
• Closed-Loop Population Management and Closed-Loop Process Improvement
• Clinical Intelligence Plus Complex Event Processing Drives Process Definition Execution
• Clinical Dashboard Displays Patient On-Protocol/Compliant, Measured, Controlled Percentages
• Patient List Manager Enables Ad-Hoc and Policy-Based Intervention Planning
• Patient Class Event Hierarchy Intermediates Patient Event Stream and Automated Workflow
• Key Performance Indicators and Process Mining Flag and Identify EMR / EHR Workflow Bottlenecks
• Individual EMR / EHR Workflow Steps are Time-stamped and Logged
• Process Mined Workflow/Event Logs Generate Detailed Process Models
• Active Clinical Intelligence and Systematically Improvable Clinical Processes Require Process-Aware Foundations
• Seven Advantages of Process-Aware EMR / EHR BPM Over Process-Unaware Alternatives
• Epilogue: EMRs / EHRs Need to Perceive and Respond to Clinical Threats and Opportunities in Real-Time
• Addendum: Glossary of EMR / EHR Workflow Terminology

Short “Editorials” on EMR, EHR, BPM, BI, CI, CEP, Productivity and Usability

• EMR, Electronic Medical Record
  • Perhaps the simplest definition of EMR is a “computerized ’systematic documentation of a single patient’s long-term individual medical history and care’” where all the words between the single quotes are simply the definition of a “medical record” (wiki)
• EHR, Electronic Health Record
  • Some use “EMR” and “EHR” synonymously. I often do. Others regard an EMR as being within a healthcare organization, such as medical office or hospital, while an EHR is a sum of capabilities to share and coordinate data and care across organizations. I’m OK with that too, but note use of the EHR acronym did not take off until the Federal government began using it instead of EMR. I sometimes hedge my semantic bets by using the phrase “EMR / EHR”. If a reader thinks they mean the same thing, then I appear to be noting synonymy. If they believe EMR and EHR mean different things, then I appear to refer to the totality of EMR plus EHR. Either works for me.
• Workflow Management Systems, Business Process Management, BPM
  • As noted in Wil van der Aalst’s 2004 book Workflow Management: Models, Methods, and Systems, by analogy a workflow management system is to a workflow system much as a database management system is to a database system (for more on this distinction). In each case, the former creates and manages the latter. Workflow management systems are narrower in scope than business process management systems, sometimes designed to do little more than flexibly automate collections of tasks. Business process management systems, or suites, add capabilities from business activity monitoring and business intelligence to process mining and simulation to flexible user-customizable user interfaces.
• Business Intelligence
  • Business intelligence, or BI, means different things to different people. Wil van der Aalst, in his recent book Process Mining: Discovery, Conformance, and Enhancement of Business Processes, notes:

“There is no clear definition for BI. On one hand it is a very broad term that includes anything that aims at providing actionable information that can be used to support decision making. On the other hand, vendors and consultants tend to conveniently skew the definition towards a particular tool or methodology. Clearly, process mining can be seen as a new collection of BI techniques. However, it is important to note that most BI tools are not really “intelligent” and do not provide any process mining capabilities. The focus is on querying and reporting combining simple visualization techniques showing dashboards and scorecards….Under the BI umbrella many fancy terms have been introduced to refer to rather simple reporting and dashboard tools.” (p. 21)

“Many vendors offer Business Intelligence (BI) software products. Unfortunately, most of these products are data-centric and focus on rather simplistic forms of analysis….process-centric, truly “intelligent” BI is possible due to advances in process mining.” (p. 261)

“BI products do not show the end-to-end process and cannot zoom into selected parts of this process….Another problem of mainstream BI products is that the focus is on fancy-looking dashboards and rather simple report, rather than a deeper analysis of the data collected. This is surprising as the “I” in BI refers to ‘intelligence’.” (p. 263, emphasis in original)

Also see How Process Mining is Related to BI where they write:

“The added value of process mining over traditional BI reporting tools lies in the depth of the analysis.

Traditional BI reporting tools focus on the display of Key Performance Indicators (KPIs) for executives in the organization. For example, the cycle times of a customer-facing process may be key in meeting certain service levels that have been agreed.

If the cycle times are out of the acceptable bounds, dashboards can highlight this problem. However, they cannot do much to uncover the root causes for this problem. Process mining can help to provide much deeper insight into the actual processes by uncovering the process flows and bottlenecks based on existing IT logs in a bottom-up manner.

Essentially, BI assumes that the underlying processes are known. Process mining takes the stand that even well-defined processes usually don’t go as planned and need to be brought into light objectively.” (my emphasis)

Out-of-bound KPI cycle times explained by process mined bottlenecks…this is exactly the capability demonstrated by one of the two EMR / EHR BPM modules presented below.

• Clinical Intelligence
  • Definitions of clinical intelligence also vary according to vendor and consultant, tool and methodology. At this point, it is perhaps best defined as business intelligence tools and methods applied to patient care and health, and left at that. Later I’ll describe a specific clinical intelligence tool presented at MedInfo2010 (slides and notes also below).
• Clinical Groupware
  • Clinical groupware is a combination of the “intentional care team processes and procedures pertaining to the observation and treatment of patients plus the tools designed to support and facilitate the care team’s work.” Note the emphasis on “team”. It’s unusual to see that same word, especially a noun, used more than once in a definition, so it must be important! I sometimes refer to clinical groupware as teamware (as opposed to “singleware”). Clinical groupware includes workflow systems, workflow management systems, business process management systems, and adaptive case management systems when applied to clinical coordination and collaboration.
• Process-Aware Information Systems
  • PAISs, or Process-Aware Information Systems, include business process management systems (which in turn include workflow management systems). While database systems and email programs, for example, may execute steps in a process, they do not contain, consult, or are “aware” of, any explicit process models. Most current traditional EMR / EHR systems are not process aware. While EMR / EHRs are gradually incorporating more-and-more sophisticated task management features, most of these capabilities are relatively frozen, their workflow not amenable to editorial control by EMR / EHR users.
• Process Mining
  • Since I own Process Mining by, arguably, the world’s expert on the topic, I might as well just quote Wil van der Aalst again:

“The goal of process mining is to use event data to extract process-related information, e.g., to automatically discover a process model by observing events recorded in some enterprise system.” (Process Mining: Discovery, Conformance and Enhancement of Business Processes)

“The healthcare industry includes hospitals and other care organizations. Most events are being recorded (blood tests, MRI scans, appointments, etc.) and correlation is easy because each event refers to a particular patient. The closer processes get to the medical profession the less structured they become. For instance, most diagnosis and treatment processes tend to be rather Spaghetti-like…. Medical guidelines typically have little to do with the actual processes. On the one hand, this suggests these processes can be improved by structuring them. On the other hand, the variability of medical processes is caused by the different characteristics of patients, their problems, and unanticipated complications. Patients are saved by doctors deviating from standard procedures. However, some deviations also cost lives. Clearly hospitals need to get a better understanding of care processes to be able to improve them. Process mining can help as event data is readily available. (emphasis in original)

I’ll illustrate the use of process mining to generate a process model for comparing nine busy pediatric practices in slides and speaker notes below.

If you are interested in process mining applied to healthcare, a good place to start are these three recent introductory posts.

Four Challenges for Process Mining in Healthcare

Process Mining in Healthcare - Case Study No. 1

Process Mining in Healthcare - Case Study No. 2

• Usability, Human Factors
  • Usability is “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” EMR / EHR usability, applying human factors principles and methods to EMR / EHRs, is a hot topic because physicians are “resisting” adoption of EMR / EHRs and there must be a good reason. The current reason célèbre is that EMRs are “clunky” and that usability engineers can fix this clunkiness. I’m a great fan of the cognitive science behind usability, but I do have a bone to pick (hmm, more like axe to grind). The relationship between EMR / EHR usability and workflow is both profound and misunderstood. When users complain about EMR usability they are often really complaining about EMR workflow that gets in their way, instead of paves their way. The biggest reason that current traditional-style EMRs are difficult to use is not so much because they weren’t created by an army of usability engineers, but because EMR / EHRs don’t rely on executable process models under editorial control (at both design- and run-time) of the users relying in them. It’s the users who should make their EMRs more usable, not EMR programmers or even usability experts. If non-programmer users could more easily push-pull-poke malleable EMR workflow into more usable workflow (for them, their goals, and their context), more usable EMRs and EHRs would result. I am *not* against usability testing. Usability testing *will* result in more usable EMRs. However, the most important form of EMR / EHR usability is a kind of meta-usability: usability of the tools users need to improve EMR usability, themselves. The most important meta-usability is the ability to improve the usability of EMR workflow. All the usability testing in the world won’t convert current document-based, data-centric, non-process-aware EMRs into workflow-based, process-aware EMRs. It’s too much of a paradigm shift and there is too much investment (and therefore design inertia) built into current legacy EMR / EHR product infrastructure.
• Productivity
  • There’s also been a lot of press about current EMRs / EHRs reducing physician productivity (this letter to the New York Times is typical). Much of the current impetus to improve EMR usability comes from this press. Yes, usability is part of the problem. However, a traditional approach to EMR usability focuses on solitary users in front of solitary computers accomplishing (relatively) solitary tasks. The alternative is to study teams of users coordinating accomplishment of coordinated tasks “in the wild” (a reference to one of the first books about distributed cognition). The problem with the first approach is that it’s difficult to generalize from simulated laboratory experiments with individual users back to the real world. On the other hand, studying teams using groupware in the real world is fraught with its own problems. However, studying how better workflow leads to better usability and higher productivity will require it. We need hard data (such as, for example, process models generated from individually time-stamped user clicks) from teams of users actually using EMRs in the real world. Process mining can provide this. In fact, process mining is already used to study usability.
• Complex Event Processing, Event-Driven Architectures
  • An event is a change in state, such as a patient who gains weight and moves from obese to morbidly obese state categories. A complex patient event is a pattern of detected events amidst a patient event stream (such as moving from obese to morbidly obese combined with being diabetic). Complex event processing, implemented in conjunction with a BPM system, provides means to react to events in real-, or almost real-, time. In the case of a process-aware information systems such as EMR / EHR workflow management (or business process management) systems, patient events can drive automated clinical workflows (such pushing action items to worklists) via workflow engines executing process definitions (more below).

Abstract, Slides, and Speaker Notes:

“Process-aware EHR BPM Systems:
Two Prototypes and a Conceptual Framework”

Abstract

Systematic methods to improve the effectiveness and efficiency of electronic health record-mediated processes will be key to EHRs playing an important role in the positive transformation of healthcare. Business process management (BPM) systematically optimizes process effectiveness, efficiency, and flexibility. Therefore BPM offers relevant ideas and technologies. We provide a conceptual model based on EHR productivity and negative feedback control that links EHR and BPM domains, describe two EHR BPM prototype modules, and close with the argument that typical EHRs must become more process-aware if they are to take full advantage of BPM ideas and technology. A prediction: Future extensible clinical groupware will coordinate delivery of EHR functionality to teams of users by combining modular components with executable process models whose usability (effectiveness, efficiency, and user satisfaction) will be systematically improved using business process management techniques.

Keywords: EMR, Electronic Medical Record, EHR, Electronic Health Record, WfMS, Workflow Management Systems, Business Process Management, BPM, Business Intelligence, BI, Clinical Intelligence, Clinical Groupware, PAIS, Process-Aware Information Systems, Process Mining, Usability, Human Factors, Productivity, Complex Event Processing, CEP, Event-Driven, Clinical Quality Measures, Protocols, Guidelines, Compliance, Outcomes, Population Health Management, KPI, Key Performance Indicators, Closed-loop Patient Care

Thank you for attending this session on Process-Aware EHR Business Process Management Systems: Two Prototypes and a Conceptual Framework.

My presentation outline is as follows…

I’ll…

• introduce EMR / EHR productivity, which can be systematically improved with business process management technology.
• discuss a prototype clinical intelligence BPM module, called PROCARE, intended to systematically improve the state of health of patients in an EHR database.
• describe a prototype process mining BPM module, called PROCESS, intended to systematically improve EHR workflow efficiency.
• Finally I’ll list seven general advantages of process-aware EHR BPMS systems over EHRs that lack workflow engines, process definitions and the functionality these enable.

The red numbered bullet points (2, 2.a-d, 3, 3.a, 3.b) correspond to upcoming numbered slides.

EHR Productivity = Information Value / Information Cost

The concept of EMR / EHR productivity can bridge between EHRs and BPM technology realms. EHR productivity is the value of information contained in an EHR divided by the cost of obtaining that information. Information value and information cost can be systematically improved, maximized and minimized respectively, using BPM techniques such as business intelligence, process mining, and complex event processing.

Closed-Loop Optimization can Systematically Improve EHR Productivity

I am sure you are familiar with negative feedback loops such as implemented by thermostats operating to minimize the difference between observed and desired temperature. The difference between observed and desired state/output steers system state/output toward desired state/output. Many complex systems, from missiles to reactors, use sophisticated implementations of negative feedback loops to optimize system behavior.

[See Closed-Loop Strategies for Patient Care Systems for further overview of closed-loop control and its history, use, and future potential in healthcare.]

Healthcare information technology increasingly seeks to implement closed-loop systems, using estimated measures of clinical outcome and resource consumption to improve performance.

In order to systematically improve EHR productivity the information value numerator should be systematically increased while the information cost should be systematically decreased. In our formulation information cost is inversely proportional to efficiency level.

Closed-Loop Population Management and Closed-Loop Process Improvement

This slide is a graphical representation of an outline to this presentation. The numbered red boxes (2, 2.a-d, 3, 3.a, 3.b) correspond to upcoming slides.

You can think of improving EHR productivity in terms of negative feedback loops within a negative feedback loop. Two inner feedback loops implement systems for systematically increasing EHR information value and systematically decreasing EHR information cost. The outer feedback loop systematically increases the ratio of EHR information value to its cost to create.

Clinical Intelligence Plus CEP Drives Process Definition Execution

PROCARE, PROvision-based Clinically Active Reporting Environment, was a prototype BPM clinical intelligence module created to interact with an EMR / EHR workflow management system that relies on a workflow engine to execute process definitions.

A clinical intelligence reporting system without ability to trigger automated workflow is a passive reporting system (in which reports must be handed to staff for disposition, “Please put a note in each patient’s chart so that the next time they have an appointment…”). An active reporting system feeds directly back to a workflow engine executing clinical process definitions to automatically perform useful tasks–hence the “Active” in PROCARE’s Provision-based Clinically Active Reporting Environment.

“Provision” is borrowed from legal terminology. It means forward-looking restriction or qualification in a contract or agreement. For example, a patient can be in a predefined class of patients provided they meet that class’s predefined criteria (age between 0 and 18, BMI > 30, etc.). Many clinical intelligence reporting systems use predefined, or user defined, criteria to include or exclude patients from numerators and denominators of clinical performance measures.

To implement an EMR / EHR-based population health management you need a measure of health state, or a surrogate such as clinical performance (those clinical quality measures, with their numerators and denominators and exclusion categories and so forth). This direct or indirect measure (or combined measures) is compared to a goal value. The difference, or at least direction, is used configure states, events, policies, and process definitions to use patient events to drive automated workflows improving health state/clinical performance. A human user, reacting to patient state reports and clinical dashboards, provides an important part of this negative feedback loop.

Clinical Dashboard Displays Patient On-Protocol/Compliant, Measured, Controlled Percentages

This slide shows a clinical dashboard relative to its function within this negative feedback formulation (”Health Monitor” in red box).

PROCARE’s clinical dashboard displays KPI’s (Key Performance Indicators) for each measure of clinical performance four numbers corresponding to the four levels of a “patient class event hierarchy” (which I’ll display several slides from now):

• number of patients in the class for which the measure applies,
• percentage of patients in each class that are compliant with a predefined protocol,
• percentage of patients for whom appropriate and timely measurements are available, and
• percentage of patients for whom observed measures are controlled (within target normal limits).

These colorful graphs on the right represent the same information in comparison to goal thresholds:

• Green means performance measure above threshold
• Yellow means at or near clinical performance threshold
• Red means below clinical performance threshold.

Patient List Manager Enables Ad-Hoc and Policy-Based Intervention Planning

Selecting a measure of clinical performance in the summary display brings up a patient list management screen for intervention planning. Creation or refinement of automated workflow policies link patient class events to automated workflows. For example, process definition steps could include sending work items to roles or users, work items that appear when the patient next is physically present, instructions that appear automatically whenever a patient chart is opened, or messages to external systems that trigger email or phone calls.

While interventions can be triggered manually (from the patient list manager for individual patients or groups of patients meeting prior or user specified criteria) or automatically (via clinical CEP policies linking patient events to automated workflows) one optimization goal is to gradually replace manual interventions with automatic policy-based interventions, decreasing resource consumption while increasing predictability. Over time, slow, inconsistent, manual workflow “compiles” into fast, consistent, automated workflow.

Linking a clinical business intelligence system and a workflow engine automatically executing process definitions is what makes PROCARE an example of clinical complex event processing. A non-programmer, a clinical user, can create and edit patient state definitions, policies linking patient events (changes in patient state), and executable process definitions, turning an EMR / EMR, with a workflow engine, into an active clinical assistant, tirelessly working to achieve goals its human users program into it.

Patient Class Event Hierarchy Intermediates Patient Event Stream and Automated Workflow

Now we’ll drill down into the patient class event hierarchy (red box labeled Patient Class Events) used to trigger automated EHR workflows.

This decision tree is the critical intermediate representation mediating between low level patient events (state changes) and higher level concepts clinical concepts such as “on-protocol,” “compliant”, “measured”, and “controlled.”

Here you can see where the numbers in the PROCARE clinical performance dashboard come from:

• #P+EHR, number of patients in EHR;
• #P+MC, number of patients meeting clinical criteria;
• #P-MC, number of patients not meeting clinical criteria;
• #P+P/C, number of patients on-protocol/compliant,
• #P-P/C, number of patients not on-protocol/compliant;
• #P+MM, number of patients for whom target metrics have been measured within specified time interval;
• #P-MM, number of patients for whom target metrics have not been measured within specified time interval;
• #P+C, number of patients for whom target measures have been measured within specified time interval and are under control (within normal limits);
• #P-C, number of patients for whom target measures have been measured within specified time interval and are not under control (not within normal limits).

Regarding #P+/-P/C (Patients On Protocol/Compliant, or not), if you have direct compliance-relevant data feeds from devices in the home, for example, this patient class event hierarchy likely should separate into On Protocol (#P+/-P) and Compliant (#P+/-C) levels.

To summarize…

Execution of appropriate automatic policy-based workflows (in effect, intervention plans),

• for patients who aren’t on protocol but should be,
• aren’t being measured but should be,
• or whose clinical values are not-controlled,

moves patients from

• off-protocol to on-protocol,
• non-compliance to compliance,
• unmeasured to measured, and from
• uncontrolled to controlled state categories,

improving individual and collective patient health state, causing a shift from red to yellow to green graphical indicators on the clinical dashboard.

KPIs and Process Mining Flag and Identify EMR / EHR Workflow Bottlenecks

Now let’s take a look at the denominator of the EHR productivity formula. This module was called PROCESS, for PROcess Comparison for Efficient System Specification.

PROCESS is an example of process mining. Process mining generates process models from workflow, or event, logs. An event data point can be as little as a number identifying a patient encounter, the name of a task (”Record Allergies”), and a time stamp.

We needed a measure of global efficiency to optimize. We chose average throughput time, also called cycle time by industrial engineers. [Note red box, "Efficiency Monitor".]

We process mined the workflow logs of nine pediatric practices to compare productivity measures and workflows, and highlight possible bottlenecks that could be alleviated by changing executed process definitions–hence the “Comparison”, “Efficient”, and “Specification” in PROCESS’s Process Comparison for Efficient System Specification. We benchmarked practice throughput volume and times against each other. Three practices stood out [see circled practices in previous slide]. We noticed that one of the three busiest pediatric practices (Practice 9, in blue) had a dramatically longer throughput time. Practices 5 (red) and 7 (green) took only 23 minutes and 44 minutes, respectively, to open and close a patient chart. In contrast, practice 9 took over eight hours to complete its charts. Obviously the patient was long gone by then.

Now we’ll drill down into a process model generated from the combined workflow/event logs of these nine pediatric practices. [Note red box, "Compare Processes".]

Individual EMR / EHR Workflow Steps are Time-stamped and Logged

This EHR workflow management system has screens devoted to each possible data review and entry and order entry step. Table letters A through Y index the names of EMR screens: Allergies, Anticipatory Guidance, Chart Review and so on. Let me draw your attention to two pairs of task screen steps, Get Patient (H) and then Current Meds (E) versus Examination (F) and then New Note (G). In the former case (H to E) a nurse gets the patient and then asks about current medications. In the latter case (F to G) a patient examination is followed by creating a new note about the patient.

Process Mined Workflow/Event Logs Generate Detailed Process Models

This is the transcribed result of process mining the workflow logs for nine pediatric practices for the busy month of October 2008. The letters correspond to the individual screen tasks. Reviewing the process models revealed that practice 9 differed from practices 5 and 7 primarily in that many charts appeared to pile up between the Examination and a New Note steps and then stay there (red arrow from F to J). Practices 5 and 7 also showed some degree of congestion earlier in their workflow (red arrow from F to G), but this apparently did not have a dramatic impact on throughput time. The practice skills instructor took one look at this and said “Who is practice nine? They are doing something wrong and I need to fix their workplans!” (Process definitions are called “workplans” in this EHR workflow management system.)

[Recall that second quote about process mining and business intelligence? PROCESS is an example of what it described, explaining an out-of-bounds cycle time (the KPI, or key performance indicator) via a potential bottleneck in the process model generated by process mining EMR / EHR event data.]

Active Clinical Intelligence and Systematically Improvable Clinical Processes Require Process-Aware Foundations

Business process management systems, or suites, rely on workflow engines and process definitions but add additional value, such as user-friendly user interfaces or visual analytics to better understand processes. More recently, in the US, the phrase “clinical groupware” has also become popular. Workflow systems are classical examples of groupware. So, in keeping with recent trends in both the health information technology and business process management industries, I sometimes refer to these systems as “process-aware clinical groupware.” What all these technologies have in common is that they are “process-aware.”

Without a process-aware foundation, that is, without an executable process model, neither PROCARE nor PROCESS (or EMR / EHR BPM modules similar to them in functionality) would be possible and have practical effect. Actionable clinical intelligence, that is, active as opposed to passive clinical reporting requires some means to automatically detect salient patient events and then automatically trigger automated workflows, transparently and usably at physician behest. These automated workflows interleave with other workflows, manual and automated, to generate a deluge of time-stamped data, the basis for generating sophisticated operational clinical process intelligence to explain and improve clinical processes. Process mining is not just about improving efficiency. Any KPI–clinical outcomes; practice productivity and profitability; patient and user satisfaction–can be compared across medical practices and difference in KPI values (good or bad) explained by processes generating or influencing them. These fact- and process-based explanations can direct further investigation and intervention.

Seven Advantages of Process-Aware EMR / EHR BPM Over Process-Unaware Alternatives

Process-aware EMR / EHR workflow management systems/business process management systems have numerous advantages over their process-unaware cousins.

Non-process aware EHRs do not distinguish between unitary tasks at the same fine degree of granularity as process-aware EHRs. Traditional EHRs often have high resolution screens with a multitude of simultaneous data review and entry and order entry options. Multiple user events, spanning multiple tasks, are often committed together to the underlying database, conflating together logically separate workflow steps. In contrast, a process definition-driven EHR can present just the data review and entry and order entry options on each screen that are relevant to a single step in a task workflow sequence. For example, a nurse checking allergies and then current medications are two different tasks that at highly granular resolution should be distinct and acquire different time stamps.

Non-process aware EHRs do not capture all the potential meaningful timestamps for those events that they do log. They may log when data and orders are committed to a database but they do not typically log when tasks are first available to be accomplished, when they begin, when they complete, and other relevant timed-stamped events such as cancellation, postponement, or forwarding. Much of this missing temporal information is invaluable for understanding why bottlenecks occur, why certain tasks are subject to rework, and what slack resources are available elsewhere in the system.

Non-process aware EHRs, even if their event logs result in useful process models and actionable insights, lack means to actively influence changes to workflow. There are no process definitions or workflow engines to execute them; so there are no process definitions to change and thereby influence and improve effectiveness and efficiency. With respect to EHR effectiveness, a clinical intelligence reporting system without ability to trigger automated workflow is a passive reporting system (in which reports must be handed to staff for disposition, “Please put a note in each patient’s chart so that the next time they have an appointment…”). A more active clinical intelligence reporting system feeds directly back to a workflow engine and process definitions to automatically perform useful tasks. With respect to EHR efficiency, even if a process model has an obvious flaw, there is no way to consistently and automatically deflect behavior at critical process junctures in order to improve throughput and throughput time.

In summary, compared to process-aware EHR workflow management, or business process, management systems, traditional EHRs (1) do not track tasks at a sufficiently high degree of resolution, (2) do not distinguish among the large number of possibly useful time-stamped events, and (3) have no means for process model insights to drive improvement at the point-of-care through automated workflow.

The next four advantages of process-aware EHR BPM systems (or process-aware clinical groupware, if you will) are generally acknowledged advantages of BPM systems over non-BPM systems.

• EHR BPM systems can be used to model and understand workflow,
• coordinate patient care tasks handoffs,
• monitor task execution in real time, and
• systematically improve clinical workflow and outcomes.

The next step in the evolution of ambulatory EMRs is squarely at the intersection between two great software industries: electronic health record systems and workflow management/business process management systems. The hybrid EMR workflow systems that result will be more usable and more systematically optimizable than traditional EMRs with respect to user satisfaction, clinical performance, patient satisfaction, and practice profitability.

Thank you!

Epilogue: EMRs / EHRs Need to Perceive and Respond to Clinical Threats and Opportunities in Real-Time

Referring back to van der Aalst’s quote about business intelligence, he clearly considers process mining to be an example of sophisticated operational business intelligence. Just as clearly, therefore, both the numerator (PROCARE) and the denominator (PROCESS) in the EMR / EHR productivity ratio are examples of clinical business intelligence/clinical intelligence. The difference between this formulation of EMR / EHR-mediated business intelligence and most other formulations is the important role of an executable process model yoked to clinical complex event functionality. Without both capabilities–to both perceive and react to clinical threats and opportunities in real-time–transparently and under flexible human control, EMRs / EHRs will not become capable of automatable closed-loop patient care or its systematic improvement.

Addendum: Glossary of EMR / EHR Workflow Terminology

A.S. Check out the EHR.BZ REPORT on Workflow, Usability and Productivity.

June 7th I attended the well-run and thought-provoking NIST EHR Usability workshop in Gaithersburg, Maryland (http://ehr.bz/nistux). I tweeted my notes. I’ve tried this before with mixed results (Dr. G’s Workflow Management EMR presentation at HIMSS, Tweeting Live from HIMSS, Tweeting Live from Process.gov).

I decided to try again!

So I…

• Tweeted my “notes.”
• Copied the tweets from Twitter.
• Reversed their order (”tail -r tweets.txt” for you UNIX folks).
• Pasted them into Wordpress.
• Edited for readiblity.
• Added more thoughts and material such as links and specific slides.

This is a long (and sometimes meandering and ruminative) document, so here is a table of contents if you’d rather proceed directly to one or other presentation topic.

• Why is Improved Usability of EHRs Important?
• The Promise of EMRs
• NIST EHR Usability Program
• Resources for the Regional Extension Centers
• Evidence-based Usability Guidelines for Promoting Safety and Efficacy
• TURF – A Unified Framework for Defining, Evaluating, Measuring, and Designing EHR Usability
• Overview of HIMSS Usability Taskforce Initiatives
• Safety, Usability and User Interface Standards in the NHS (Virtual Presentation from the UK)
• Human Factors Approaches to Improving EHR Usability
• Collaboration and Consensus through Standards – The National Technology Transfer and Advancement Act
• A Community Approach to EHR User Experience Measurement
• Community of Profession Model
• Human Factors / Usability for Medical Devices at FDA: An Historical Perspective
• Building More Usable EHRs – Supporting the Needs of Developers “Focus on Faster & Usable Clinical Documentation”
• Developers: Supporting the Needs of Patients
• Educate, Motivate, and Improve: In Favor of Inspecting and Rating UCD/Usability Processes
• Usability is the Key to Stimulating EHR Innovation and Adoption
• Promoting Usability in Healthcare Organizations with a New Usability Maturity Model
• Guidelines for Improving Usability: Proposed EHR Usability Evaluation Protocol
• Government Best Practices in System Usability: Brief History & Status
• The Relationship between Health IT Usability and Patient Safety: Towards an EHR Usability Safety Framework
• Audience Questions/Comments During Technical Feedback on EUP (EHR Usability Protocol)

I should make a disclaimer. I am biased. Lack of EMR usability has more do with document-based versus process-based approaches to building EMRs than it has with government versus industry approaches to driving EMR innovation. Most current EMRs rely on structured documents and unstructured processes. Until EMR users are given tools to model, execute, monitor and systematically improve the standardizable processes generating the structured documents (and potentially redirect those processes on the fly), lack of usability will continue to slow EMR adoption. (See EHR/EMR Usability: Natural, Consistent, Relevant, Supportive, Flexible Workflow) Whether government or industry or both accomplish this, and how, is an important debate, but more about ideology than what is technically retarding EMR usability.

Let the tweets begin!

c_wb Looking forward to the NIST EHR / EMR Usability workshop tomorrow

c_wb I’ve arrived at the NIST EMR / EHR usability workshop, my gadgets are charged, I have my coffee, presentations are about to begin …

By the way, I read NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records to prepare

c_wb Today I’m attending Measuring, Evaluating & Improving Usability of EHRs EMRs Workshop NIST Gaithersburg, MD

A Community-Building Workshop: Measuring, Evaluating and Improving the Usability of Electronic Health Records, June 7th, 2011, Gaithersburg, MD

Original announcement and agenda:

http://ehr.bz/nistux (cached announcement and agenda)

Opening remarks A (cached)

Welcome to: A Community-Building Workshop: Measuring, Evaluating and Improving the Usability of Electronic Health Records

Opening remarks B (cached)

A Community-Building Workshop: Measuring, Evaluating and Improving the Usability of Electronic Health Records

c_wb NIST: innovation/competitiveness by advancing measurement/standards/tech 2 enhance econ security/quality of life

Cramming NIST’s mission statement into 140 tweetable characters resulting in something a bit terse, so…

NIST’s mission:

“To promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life.”

Why is Improved Usability of EHRs Important?

c_wb Starting: Intro 2 workshop

c_wb Usability: strong/direct relationshp w/ clinical productivity, error rate, user fatigue & satisfaction

Also a bit terse, but the cool thing is that these were great search terms to find the original quote

http://www.google.com/search?q=usability+strong+direct+relationship+clinical+productivity+error+rate+user+fatigue+satisfaction

“usability is one of the major factors—possibly the most important factor—hindering widespread adoption of EMRs. Usability has a strong, often direct relationship with clinical productivity, error rate, user fatigue and user satisfaction–critical factors for EMR adoption. Clinicians lose productivity during the training days and for months afterward as they adapt to the new tools and workflow. Some productivity losses are sustained, mostly due to longer time needed for encounter documentation in complex patients” (Defining and Testing EHR Usability)

c_wb ONC goals include EHR usability transparency

There were several other goals regarding usability, but I wasn’t a quick enough so, from presentation slides distributed later…

ONC’s Goals

• Improve transparency on usability
• Promote technology that fully supports care
• Identify and address potential safety issues, but also factors that affect efficiency and effectiveness, user satisfaction, etc.
• Enable constructive innovation

c_wb #ehrusability is the hashtag for NIST EHR usability workshop

Introduction: The Promise of EMRs

Presentation PDF / Cached

@c_wb Starting: Introduction: Why is Improved Usability of EHRs Important?

c_wb one pediatric growth chart takes 8 clicks to get to open…

Wish I had a nickel for every time “click” was mentioned at this workshop. It also on the minds of folks not attending the workshop (pro and con: here, here, here, here, here and here are representative).

The number of clicks is a fairly superficial measure of (lack of) usability; three easy, fast, automatic clicks is can be more usable than one long tortuous click. However, cognitive effort is harder to measure then physical user events, so it’s a convenient surrogate. This point was made several times by members of the audience. I suspect that most people attending the workshop agree, but that “click” is shorthand for a wide variety of data and order entry costs: time (to target), effort (to find target), and error (if missed).

Perhaps entering data into an EHR/EMR should resemble playing a musical instrument. A pianist can effortlessly click on a lot of keys in a very short period of time.

c_wb poor usability can cause medical errors. We need to figure out how to test, identify and prevent them

The workshop focussed more on error minimization and patient safety than other possible usability goals such as speed, productivity and user satisfaction, though each of these topics were indeed represented in presentations and discussion. The following slide is from a later presentation. Notice that “critical errors that impact patient safety” and “errors and failures” are highlighted in red.

Overview of Current Programs for Improving EHR Usability (at NIST)

c_wb Starting: Overview of Current Programs for Improving EHR Usability

NIST EHR Usability Program

Presentation PDF / Cached

c_wb Starting: NIST EHR Usability Program

c_wb NIST funds EMR usability research, such as Human Factors Guidance to Prevent Healthcare Disparities with EHRs http://1.usa.gov/lXcPYT

Resources for the Regional Extension Centers

Presentation PDF / Cached

c_wb Starting: Resources for the Regional Extension Centers Usability of EMRs

c_wb accessibility and usability of data was a theme of 2010 HITPC Meaningful Use hearings

Themes that Emerged from 2010 HITPC/ Meaningful Use Workgroup Hearings

• Achieve 4 Es: engage, educate, empower, and enable
• Meet needs of diverse population
• Accessibility/usability of data – Need for mobile apps(esp.for vulnerable populations) – Contextualizing information – Multiple languages – Compatible with assistivetechnologies
• Patient-provider secure messaging
• Incorporate patient-generated data into EHR
• Provide ample training on all functionalities

c_wb regional extension center: need training to get to 4 clicks instead of 15

No kidding. I know of two EHRs that both allow a physician users to approve a refill request; one takes four clicks and the other takes 25 . Yeah, I know, now I seem to be contradicting myself. But in this case it’s four fast automatic effortless clicks versus 25 slow laborious clicks.

c_wb RECs to help train to “vendor specific workflow”

Interesting phrase: “vendor specific workflow”

The slide says “Work directly with providers on a regular basis. Use the HITRC for usability tools and resources. More participation with RECs; more participation with RECs; Webinars, Training, Response; Vendor specific workflow.(my emphasis)

Given that a majority of the total cost of owning an EHR / EMR can associated with workflow customization issues (I do have reference on this, but not immediately at hand), it’ll be interesting to observe how RECs cope with this particularly important and problematic aspect of implementing traditional EHRs.

See my Mirror, Mirror, On the Wall, Which EMR is Least Traditional Of All? for some tongue-in-cheek comments on how darned difficult it is to cope with the relatively uncustomizable workflow of traditional EMRs / EHRs.

Evidence-based Usability Guidelines for Promoting Safety and Efficacy

Presentation PDF / Cached

c_wb Starting: Evidence-based Usability Guidelines 4 Promoting Safety & Efficacy

c_wb took sixty yrs to get to standardized time zones, can we get 2 usability faster?

c_wb lessons other industries: apple microsoft android “usability guidelines are powerful”

Usability Guidelines are Powerful

• Apple, Microsoft, Android, …
• NASA, FAA, DoD, …
• HFES, WWW-Consortium, …
• RAISE QUALITY
  • Promote Consistency
  • User Performance Speed Development
  • Programmer Productivity Reduce Errors
  • Organization Reputation

Interesting though to contrast the slide above and the slide below (from a later presentation)!

This second slide shouldn’t be taken out of context though. It was presented as a typical criticism of government-led EMR usability test that must be acknowledge and addressed.

c_wb for a good model of research-based web design and usability guidelines see http://usability.gov

c_wb HIT usability researchers find it difficult to obtain user docs, screenshots, demos from EMR EHR vendors for research purposes

Yeah, I know what he is talking about. When I was in academia it was very difficult to get cooperation from EHR / EMR vendors to help train students let alone conduct research. When I worked for an EHR / EMR vendor I suddenly had access to a plethora of such materials, butdid not have time (or charge) to train students (as opposed to users) or conduct and present research. I started this blog in an attempt to bridge this gap and resolve (to my satisfaction) some of these inherent contradictions. I wrote about this in Walking the Fine Line between Marketing and Education.

TURF – A Unified Framework for Defining, Evaluating, Measuring, and Designing EHR Usability

Presentation PDF / Cached

c_wb Starting: TURF: Unified Framework 4 Defining, Evaluating, Measuring & Designing EHR usability

TURF = UFuRT? Below is a diagram from 2007 paper…

UFuRT: A Work-Centered Framework and Process for Design and Evaluation of Information Systems (cached)

c_wb TURF = task + user + function + representation

Note the common components between the previous and subsequent slides: task, users, function, representation, intrinsic difficulty/complexity, extrinsic usability/difficulty

c_wb “function saturation”

c_wb “overhead in designer model” less is better

c_wb measuring usability: learnability, efficiency, error prevention and recovery

How to Measure Usableness?

• Learnability
  • trials to reach a certain performance level
  • items that need to be memorized
  • sequences of steps that need to be memorized
  • Etc.
• Efficiency
  • Time on task
  • Task steps
  • Task Success
  • Mental effort
• Error Prevention and Recovery
  • Error occurrence rate
  • Error recovery rate

c_wb efficiency measure: time on task in seconds (5 minutes on CPOE )

c_wb number of steps, mental effort (thru cognitive modeling)

For an overview of cognitive modeling see Toward Cognitive Modeling for Predicting Usability

c_wb TURF in action: reduced steps from 91 to 14

Note that if an EHR does not have some means for its users to modify its workflow then this means going back to the programmers to “unfreeze”, change, and then “refreeze” its workflow. I addressed this in my post Litmus Test for Detecting Frozen EHR Workflow.

c_wb in summary: usability is definable, measurable, doable

By the way, usability approaches that emphasize an entire team as the user instead of a single user before a single display (groupware instead of “singleware”) are complementary with workflow-oriented approaches such as TURF/UFuRT. This is why, in 2004 I wrote the following about the importance of workflow management systems (today business process management systems and suites) to EHR / EMR usability:

“Workflow Management and EHR Usability

EHR workflow management concepts mesh with research initiatives to improve EHR usability. For example, Human-Centered Distributed Information Design [6] (there applied to EHR usability issues) distinguishes four levels of distributed analysis: user, function, task, and representation, which correspond well to workflow management architectural distinctions.

[1] Distributed user analysis can be interpreted to include allocation of tasks, relationship between roles, and task-related messaging, all of which are important workflow management concepts.

[2] Distributed function analysis involves high-level relationships among users and system resources. From a workflow management perspective, this includes who reports to whom and who is allowed to accomplish what.

[3] Distributed task analysis roughly corresponds to the creation of process definitions that in turn drive EHR behavior: What is to be accomplished by whom, in what order, and what needs to happen automatically.

[4] Distributed representational analysis corresponds to something that workflow management systems intentionally do not address. Workflow management system design tends to be agnostic about how information is displayed to, transformed, or collected from the user. The underlying workflow engine is intended to be a general purpose tool that can be used to sequentially launch whatever screen or initiate whatever behind the scenes action that the implementer of the workflow system deems most apt as part of workflow analysis and design. However, by remaining orthogonal to the choice of screen, by not mandating or hard coding, the designer/implementer is free to bring to bear the powers of representational analysis to use whatever screen and attendant representation is most appropriate.

Thus, workflow management concepts are consistent with human-centered distributed information design, an important emerging area of medical informatics research. “Task-specific, context-sensitive, and event-related displays are basic elements for implementing HCC [human-centered computing] systems,” (p. 46 [6]) and they are the basic elements provided by EHR workflow management systems, too.” (my emphases)

Overview of HIMSS Usability Taskforce Initiatives

Presentation PDF / Cached

c_wb Starting: Overview of HIMSS Usability Taskforce Initiatives

c_wb promotes industry education about usability principles & measurement

c_wb usability maturity model white paper

Defining and Testing EMR. Usability: Principles and Proposed Methods of EMR Usability Evaluation

c_wb subgroups include HIMSS Celltop Design Workgroup: smartphones, handheld design tenets, partnership NIH & HIMSS

c_wb next steps: maturity model checklists, attention education, more white papers

Safety, Usability and User Interface Standards in the NHS (Virtual Presentation)

Presentation PDF / Cached

c_wb starting virtual presentation from britain on NHS usability program

c_wb CUI = Common User Interface, speaker has background in aviation human factors (as have I, sort of) which is advanced over healthIT with respect to usability

c_wb common user interface example: patient banner eg “display comma after family name”

c_wb NHS lessons learned: usability + safety strongest message, considerable common user interface docs online at portal located at http://www.cui.nhs.uk

c_wb “there’s some very poor product out there” “woefully clunky”

Amen! Question: what is the best way to make EHRs generally less clunky?

• Government?
• Industry?
• Both?
• How?

Even bigger question: Is above the right question?

Human Factors Approaches to Improving EHR Usability

Presentation PDF / Cached

c_wb Starting: Human Factors Approaches to Improving EHR Usability

c_wb amusing user videos “we only have 3 screens, we don’t need 500 buttons”

The actual quote was “We only have three screens–preop, intraop, postop–we don’t need five hundred buttons” How do I know? I used the Looxcie wearable camcoder to provide a resource to consult when I want the exact wording of something.

[See post to watch Flash video]

RT amalec NHS reps bemoan lack of visibility into ehrusability; no recourse after you sign the ehr contract

c_wb +1 RT @amalec: Super cool ehrusability lab in Canada with live video recorded simulation

c_wb challenges: generalizability, resource requirements, risk vs usability, comprehensiveness…

c_wb question from audience: from an CMIO, how to tap into your (the folks presenting at NIST EHR usability conference) usability expertise?

Answer: Join this EHR usability community that NIST is in the process of creating.

c_wb question: how to make sure requirements are done right?

c_wb answer to previous question from virtual NHS folks: usability needs to be brought up in contract phase, otherwise it is two late for it to influence requirements

c_wb member of panel: patient is also a user, an example is a patient identifying an error in chart when looking at the EMR screen

As an aside, I once suggested that pediatric EMR EHR user interfaces ought to, and eventually will, evolve to look like illustrated children’s books (as part of a large exercise to make pediatric offices more-and-more child friendly). Why? Because the pediatric patient is an EMR user too. EHR buttons ought to be big enough for not only the pediatrician to hit on the fly (respecting Fitts and Hicks laws of target acquisition) but for the sharp-eyed child to see. An EMR is not just [insert standard EHR definition here], it is a form of persuasive technology. Big buttons displaying soccer balls and report cards carrying the letter “A” ought to be part and parcel of pediatric EMR user interfaces.

Cute representations of real and imaginary animals *ought* to scamper about an EMR UI, drawing in the child EHR user, not just making them less fearful, but entrancing and motivating them. Imagine a pediatrician clicking on a big animated button while saying “Come sit here and let’s see what the big blue bear thinks you should do about your cough!” EHRs, really usable EMRs, by Disney or Nickelodeon are in our future. Just not sure when–how soon or how long.

c_wb question: can NLP natural language processing help? docs wont tolerate lots of clicking

I took all the courses necessary for a degree in Computational Linguistics before switching into Intelligent Systems, including phonetics, phonology, morphology, syntax, semantics, pragmatics, and NLP I, II, and III, plus knowledge representation and NLG (natural language generation). The problem I have with replacing properly-managed structured data entry with speech recognition and natural language processing is this: a human, either the original speaker or someone else (perhaps some sort of post-editor) still needs to proof the string of linguistic tokens emitted by a speech recognition-based EMR user interface, or canonical representations of what those tokens mean.

Even if speech recognition is 99.5% correct, as long as non-automated proofing and post-editing is required, the fire-and-forget nature of clicking (or, preferably, touching) a picklist item, which does not require proofing or post-editing, is superior. And as for slowing down the physician user (the so-called “clickorrhea” I’ve occasionally tweeted about, again, if Fitts and Hicks laws are respected, structured data entry can actually be faster and more accurate than other data input modalities. Where speech recognition makes more sense, at this point in the evolution of the degree of intelligence possible by the technologies we can bring to bear on this problem, is in the mobile smartphone interface. However, here the context is severely constrained and the amount of required proofing and post editing is minimal (though still required).

By the way, computational linguistics is relevant to not just processing medical language generated by humans, but communication between EMR systems as well (especially at the higher, currently less considered, levels: pragmatics and discourse).

c_wb Answer: we’ll compare input modalities, including NLP Question: can NLP help?

RT ahier Following tweets from ehrusability workshop http://bit.ly/kexzxu

Hi Brian!

Collaboration and Consensus through Standards – The National Technology Transfer and Advancement Act

No Presentation PDF

c_wb Starting: Collaboration & Consensus thru Standards – National Technology Transfer & Advancement Act

c_wb by its very nature standards are a “collaboration” vehicles and “consensus” processes

c_wb NIST has a standard for creating standards (hmm, is there a standard for creating standards for creating standards?)

A Community Approach to EHR User Experience Measurement

Presentation PDF / Cached

c_wb Starting: Community Approach to EHR User Experience Measurement

c_wb classic tradeoff between focus vs participation: so “focussed collaboration” approach

From slide presentation speaker notes:

“Focused collaboration means that we’re engaging a broad array of stakeholders in the development process, but managing their work property to ensure the most efficient and effective process. What we don’t want to see is a high degree of focus (top down, heavy-handed government-driven process) with little participation from outside stakeholders; nor do we want a highly participatory process with no strong focus that results in a lot of great ideas, but no results. We’re seeking the best of both worlds.”

c_wb some say: if the gov created usability tests we wouldn’t have iphone or android

We saw this slide earlier

c_wb others say usability is indeed a science but that clinical workflow are “nuanced”

I presume (perhaps too much, though) that “nuanced” means not susceptible to formalization, formal analysis, or automatic execution in the service of greater data and order entry usability. I absolutely disagree.

From EMRs, EHRs, and Clinical Groupware Need to Solve “The BPM Problem”: Why Not Use BPM to Help Do So?

While it is true that most current traditional EMRs lack facility to model and execute workflow, future EMRs based on workflow management systems (WfMSs: workflow engines plus process definitions) and business process management technology (WfMS plus business intelligence, business activity monitoring, process mining, complex event processing, process simulation optimization, adaptive case management, etc.) inevitably will.

c_wb vendors don’t want government say what products are good or bad, we [vendors] don’t want the government to incent us into creating technology that stinks

c_wb BTW funny typo on slide, text was “…we want…” instead of “we [don't] want the government to incent us into creating technology that stinks”

Audience reaction to the type was good humored laughter. I certainly appreciated the lighter moment after hours of seriousness.

After the lighter moment, there is this: “So we have this classic tradeoff of the vendor community not wanting, for absolute valid reasons, the government to tell the world what products are good or bad, and the provider community saying, look we want to use electronic health records but we need to improve practice workflow, we want to make sure we have the right signals about the products we purchase”

Again, a central question appears to be: How to make EMRs, on the whole, “less clunky?

c_wb EHR EMR Usability professional: “sometimes i feel like an island in my own organization: the product is done, marketing brings me in, ‘the usability stinks’”

From my Intuitive vs. Intuitable EMRs, EHRs, and Clinical Groupware: Do We Need Smarter Users or Smarter User Interfaces?

“Usability can’t be “added” to EMRs, EHRs, or clinical groupware. It has to inform and influence the very first design decisions. And there are no more fundamental early design decisions than what paradigm to adopt and platform to use.

No matter how “intuitable,” EMRs without executable process models (necessary to perceive, reason, and act, and later systematically improve), cannot become fully active and helpful members of the patient care team. Wrong paradigm. Wrong platform.

Truly “intuitive” process-aware clinical groupware, on the other hand, has a brain, variously called a BPM, workflow, or process engine. This is the necessary platform for delivering context-aware intelligent user interfaces and user experience to the point of care. Right paradigm. Right platform.”

Community of Profession Model

Presentation PDF / Cached

Human Factors / Usability for Medical Devices at FDA: An Historical Perspective

Presentation PTT / Cached

c_wb Starting: Human Factors / Usability for medical devices at FDA: Historical Perspective

c_wb medical device milestones, 1976 bureau of MD, 1984 congress hearings on deaths

c_wb 1999 “To error is human” 98,000 deaths, 5th cause death, cost $29B

c_wb key: FDA review of pre-market submissions, outreach 2 industry

c_wb feedback: sales increase w/ satisfaction of customers for devices w attention to human factors driven by FDA

c_wb FDA concerns about device usability include relying on checklists and rating scales instead of systematic usability reviews

See my own comments about the use of checklists to evaluate EMR EHR usability

Building More Usable EHRs – Supporting the Needs of Developers “Focus on Faster & Usable Clinical Documentation”

Presentation PDF / Cached

c_wb Starting: Building More Usable EHRs: Supporting Developers “Focus on Clinical Documention” usability EMRs NIST

c_wb one study’s conclusion: “current EMRs frustrate physician collection of data”

Another study:

c_wb “the ‘we computerized the paper, so we can go paperless” fallacy” displays not as portable, flexible or well designed as paper

c_wb remove tension btwn free text vs structured documentation

This from this presentation’s key slide:

Recommendations From Literature

Remove tension between free text versus structured documentation

Clinical documentation needs to support both seamlessly

• Usability and semantic interoperability go hand in hand
• Refuse systems that do not deliver both
• Remove tension between clinician/physician documentation as a billing vehicle and as a clinical documentation tool
• Improved data input and richness of documentation can coexist if you design the system properly
• Usability is perhaps more crucial than interoperability
• The question of interoperability will be unresolved if clinicians fail toaccurately record the data

c_wb improved data input & richness of clinical documentation can coexist

Also the point of my rejoinder to the many criticisms of structured EMR EHR data entry

Developers: Supporting the Needs of Patients

No Presentation PDF (no slides were used)

c_wb Starting: Developers: Supporting Needs of Patients Usability of EHRs EMR

c_wb among users: physicians have the highest standards but also lowest tolerance (hmm, not bad, just a fact)

In fact my experience has been that part of the difficulty in developing usable EMRs has been creating an EHR that is useful and usable enough to physicians, in spite of its flaws and their high standards, to sustain physician engagement in the necessary process of improving EMR usefulness and usability.

c_wb As soon as you talk about certification your are talking about an idealized model EMR vendor customers are skeptical

c_wb “i don’t believe the price of adding government [to this mix] will be rewarded w/ better outcomes”

Again consulting my Looxcie:

“What you are saying is that your design is meaningful for every purchaser out there and your standards, your metrics, your way of saying yes we’ve have these numbers and tests are going to translate into provider happiness. As I said, I challenge the assertion that the industry has failed but I find it even more surprising the assertion we can’t do better. We think there is a role for usability in healthcare because true usability has to start out with the tasks the users have to provide, or have to perform everyday, it’s their requirements. So let’s start to look at some of those requirements that they have for meaningful use right now and figure out the way we can make those processes more efficient and more usable while still maintaining their usefulness. I think ultimately then what we need to do is to first of all provide support to those organizations like ISMT [???] and give the purchasers the tools that they need. Because we have clinicians who demand quality based on markets that have to bow to those demands in order to survive. I don’t believe the price of adding government into this dynamic will be rewarded with better outcomes. Thank you…”

Educate, Motivate, and Improve: In Favor of Inspecting and Rating UCD/Usability Processes

Presentation PDF / Cached

c_wb Starting: Educate, Motivate & Improve: In Favor of Inspecting/Rating UCD/Usability Processes

c_wb +1 “who is the user? it’s a cooperative *group* of users” [that is the user, not an individual user] #ehrusability #EMR clinical #groupware

I absolutely agree with this point. Focussing on the individual user in front of a single screen will ultimately be counterproductive. I tagged this tweet with the hashtag #groupware because I think that the phrase (and movement) associated with “clinical groupware” has this particularly right. Clinical Groupware, Care Coordination, and EMR Workflow Systems: Key Ideas provides an overview of this history of groupware as it relates to workflow (and getting workflow right is so important to usability, and, in my view, considerably misunderstood)

c_wb vendors are worried they will spend more time getting certified instead of improving usability

EMR certification does have the danger of running afoul of the famous software development “Iron Triangle”. Given the same level of resources a software product can increase one or two of 1) features, 2) quality, 3) time to market only at the expense of one of two of the remaining goals. By diverting resources to meet certification not only can one set of features (those required for certification) crowd out other features, but also constitute an obstacle to quickly getting a stable and usable EHR release to market. Usability certification may, ironically, potentially forestall the very usability innovation it seeks to advance.

c_wb summary: need incremental improvement & competitive dynamic to invent new solutions

c_wb Love “Focus on the ‘Five Big Tasks’” RT @healthfinch “War on EMR usability” Should gov get involved? post.ly/2Aagh

c_wb Agree RT @MetaMetaThinker @c_wb we should continue to automate where it makes sense and let providers focus on practicing medicine

One of the fun bonuses of tweeting notes live from a presentation is that folks (in the audience or not present) will occasionally publicly (or privately, by direct message) chime in. I’ve previously written about this Twitter-mediated conference back channel.

Usability is the Key to Stimulating EHR Innovation and Adoption

Presentation PDF / Cached

c_wb Starting: Usability is the Key 2 Stimulating EHR Innovation & Adoption

c_wb Usability: extent EMR EHR can B used by specified/S users 2 achieve S goals w/effectiveness, efficiency & satisfaction in S context NIST

It’s an interesting exercise to extend this usability definition to a clinical groupware approach to EMRs:

“Great definition…but it just seems so, well, “singleware-ish.” Clinical groupware needs a less abstract definition of usability that is more direct about groupware’s unique usability issues (see the sixth, seventh, eighth, ninth, and tenth quotes from my Clinical Groupware…Key Ideas post). How about:

Clinical groupware usability is…

“The extent to which clinical groupware can be used by specified teams of users to coordinate activity and achieve specified collections of goals with overall effectiveness, efficiency and satisfaction in specified contexts of use.”

[Those links to the sixth through tenth quotes about groupware usability? I pull that material into this post after its conclusion below.]

c_wb meaningful use has emphasized functionality over usability, 100 clicks 2 doc’mt smoking status!

Surely an exaggeration, but point well taken.

c_wb 2 types of usability: individual usability vs workflow usability, how can we reduce physician work/steps/task?

Absolutely agree. Also see material on clinical groupware usability in postscript.

c_wb Government should do 3 things: define usability measures, promote open platforms and APIs, national EHR usability database

c_wb compare users to find “positive deviance” find people doing well and find out what they are doing to learn from it ehrusability

Wrote about the potential to do something similar relative to comparing medical practice productivity measures and explaining them in terms of differences in workflow.

c_wb individual vs workflow usability: think more about how usability affects team-based care ehrusability NIST emr clinical groupware

Again, incredibly important insight, this! In my post The Cognitive Science Behind EMR Usability Checklists I wrote:

“[T]here is no guarantee that optimizing single user usability won’t in suboptimize higher level global system goals. So I prefer a definition of usability that emphasizes team, rather than individual, performance.”

Again, see postscript.

Promoting Usability in Healthcare Organizations with a New Usability Maturity Model

Presentation PDF / Cached

c_wb Starting: Promoting Usability in Healthcare Organizations with a New Usability Maturity Model

c_wb Speaker defined usability as “sexier products using a process”

Now that is a sexy definition of usability!

c_wb Usability Maturity Model: unregulated, preliminary, implemented, integrated, strategic

While we are on the topic of maturity models I’d like to mention two others 1) the Software Capability Maturity Model developed at CMU’s Software Engineering Institute (the original MM that has inspired other MMs, and 2) the Business Process Management BPM Maturity Model.

Relative to the Capability Maturity Model, Wikipedia says more generally:

“A maturity model can be viewed as a set of structured levels that describe how well the behaviors, practices and processes of an organization can reliably and sustainably produce required outcomes. A maturity model may provide, for example :

• a place to start
• the benefit of a community’s prior experiences
• a common language and a shared vision
• a framework for prioritizing actions.
• a way to define what improvement means for your organization.

A maturity model can be used as a benchmark for comparison and as an aid to understanding - for example, for comparative assessment of different organizations where there is something in common that can be used as a basis for comparison. In the case of the CMM, for example, the basis for comparison would be the organizations’ software development processes.”

Relative to a BPM Maturity Model here is a similar five level chart from “Towards a Business Process Management Maturity Model”

Why do I include these maturity models? The Software CMM is the grandaddy of maturity models and therefore important context for considering any EHR usability maturity model. And the healthcare industry has the lowest BPM maturity of any major industry segment. At least some of the (lack of) usability issues afflicting EHRs is that they are not process-aware in the sense as workflow management systems and business process management suites.

c_wb Launching usability: wake-up calls, individual infiltration, internal champion, external experts

Guidelines for Improving Usability: Proposed EHR Usability Evaluation Protocol

Presentation PDF / Cached

Presentation PDF / Cached

c_wb Starting: Guidelines for Improving Usability: Proposed EHR Usability Evaluation Protocol

c_wb EHR Usability Protocol focuses on *most critical* issues first, others later

Presumably preventing errors threatening patient safety

From the slide:

EHR Usability Protocol (EUP)

• The EUP provides a methodology for identifying and eliminating risks to patients due to poor user interface design.
• This focus is the foundation of many existing, validated protocols for evaluating the usability of systems where safety is a critical component of user operation.
• EUP focuses on the most critical issues first.
• Other dimensions of usability are important.(my emphasis)

My interpretation is that while speed, productivity, profitability, satisfaction, engagement, etc. are important and may eventually be tackled, that patient safety is paramount and therefore will drive creation of the EHR Usability Protocol.

c_wb objectives: eliminate “never events”, ID/prevent critical use errors…

c_wb objectives:… ID areas for improvement and report in Common Industry Format CIF

c_wb EUP does not describe “look & feel” & therefore will not discourage innovation

Is it possible to improve usability without changing the look and feel of a user interface? From the Wikipedia entry on “look and feel”:

“Look and feel in operating system user interfaces serves two general purposes.

First, it provides branding, helping to identify a set of products from one company.

Second, it increases ease of use, since users will become familiar with how one product functions (looks, reads, etc.) and can translate their experience to other products with the same look and feel.” (my emphasis)

c_wb Errors of commission vs omission (harder to detect)

c_wb Never events: commission, ommission wrong med, [2 more didn't get!]

Here’s the complete list (from the slide):

“Never Events

The proposed categories of never events are:

• Wrong patient action of commission event: Actions with potentially fatal consequences are performed for one patient that were intended for another patient because two patient identifiers were not displayed in an area of the screen that is visible without scrolling
• Wrong patient action of omission event: A patient is not informed of the need for treatment because the wrong patient’s name was displayed on clinical data for another patient
• Wrong medication event: A patient receives the wrong medication, dose, or route because the displayed information was not accurate or required viewing information on hidden screens to be accurate
• Delay in care event: A patient should not receive a life-threatening delay in the provision of critical care activities due to design decisions made for administrative, billing, or security objectives
• Unintended care event: A patient should not receive unintended care actions due to actions taken to test software, train users, or demonstrate software to potential customers.”

c_wb More errors: sequence & timing errors (both subclasses of errors of commission)

c_wb Quant vs qual/attitude vs behavior: summative usability testing is quantitative results from behaviors

(red circle added)

c_wb “discount usability testing” not repeatable across multiple designs

(red highlights in original)

Had to look that phrase up–Jakob Nielson popularized the phrase and idea of discount usability engineering, about which he wrote:

The Discount Usability Engineering Approach

“Usability specialists will often propose using the best possible methodology. Indeed, this is what they have been trained to do in most universities. Unfortunately, it seems that “le mieux est l’ennemi du bien” (the best is the enemy of the good) [Voltaire 1764] to the extent that insisting on using only the best methods may result in having no methods used at all. Therefore, I will focus on achieving “the good” with respect to having some usability engineering work performed, even though the methods needed to achieve this result are definitely not “the best” method and will not give perfect results.

It will be easy for the knowledgable reader to put down the methods proposed here with various well-known counter-examples showing important usability aspects that will be missed under certain circumstances. Some of these counter-examples are no doubt true and I do agree that better results can be achieved by applying more careful methodologies. But remember that such more careful methods are also more expensive — often in terms of money, and always in terms of required expertise (leading to the intimidation factor discussed above). Therefore, the simpler methods stand a much better chance of actually being used in practical design situations and they should therefore be viewed as a way of serving the user community.”

c_wb usability testing process: kickoff/discovery, preparation, data collection, analysis/reporting

c_wb key differences summative testing for EHRs: requires more moderators with greater expertise & more tasks mandated by meaningful use than non EMR testing

(red highlights in original)

c_wb many are tasks tied to MU meaningful use criteria

c_wb test administrators will need advanced degrees in human factors and minimum 3 years experience

c_wb Next steps: protocol development, test protocol examples, data sheets, develop more specific tasks

Government Best Practices in System Usability: Brief History & Status

Presentation PDF / Cached

c_wb Starting: Government best practices in system usability: Brief history & status

c_wb Human factors in design of safety critical systems: Book: The Chapanis Chronicles: 50 years of HF Research, education & design

Great to see the well-reviewed biography of aviation human factors/ergonomics researcher Alphonse Chapanis. I became peripherally aware of Chapanis when I took a course at the University of Illinois Institute of Aviation (home of U of I’s Human Factors and Ergonomics program). I’ve frequently cited his contemporary, Paul Fitts, relative to Fitts Law and the need for big buttons for EHRs. I even wrote a poem about him.

“The early educators in the field-Alex Williams, Al Chapanis, Paul Fitts, Ross McFarland, Len Mead, Lick Licklider, Neil Warren, John Lyman, Jack Adams, George Briggs, and Ernest McCormick-had in common a recognition of the importance of a multidisciplinary approach to aviation problems, and their students were so trained.” (The Adolescence of Aviation Psychology)

As we are currently in the adolescence of “EMR Psychology”, I think there is indeed a historical model in aviation psychology to inspire and guide us today.

c_wb Human factors was born 70 years ago in the aviation industry when planes were falling out of the skies

I enjoyed seeing connections drawn between the history of aviation human factors and EMR / EHR usability. For more on the subject see EHR/EMR Workflow System Usability–Roots in Aviation Human Factors.

c_wb Notable UI Incidences: NORAD false alarms, 3 Mile Island, Flight 965 Cali CA

• NORAD false alarms
• Three Mile Island
• Flight 965 Cali CA

c_wb NIST involved in refinery user interface: reorg control room operator info, reduced plant incidents to one third of previous

c_wb FAA Order 9550.08*: “human factors shall be systematically integrated…all FAA elements & activities

FAA order 9550.08*

“Human factors shall be systematically integrated into the planning and execution of the functions of all FAA elements and activities associated with system acquisitions and system operations. FAA endeavors shall emphasize human factors considerations to enhance system performance and capitalize upon the relative strengths of people and machines.”

c_wb Human factors @ Dept of Defense: Human System Integration/Manpower Personnel Integration (MANPRINT, I kid you not)

The Relationship between Health IT Usability and Patient Safety: Towards an EHR Usability Safety Framework

Presentation PDF / Cached

Presentation PDF / Cached

c_wb Starting: Relationshp btn HealthIT Usability/Patient Safety: 2wards EHR Usability Safety Framework

c_wb NIST working on Usability-Safety Framework max benefits 2 users & patients & min harm

I like this diagram and look forward to where it leads. What I like most is that it is “closed loop” and appears amenable to a “process-aware” approach (process design, implementation, enactment and diagnosis) to systematic optimization (minimization of error, maximization of safety).

c_wb Use Errors: Patient ID, mode, data accuracy, visibility, consistency, recall, feedback, data integrity

c_wb Evaluation indicators: workarounds, redundancies, burnout

c_wb Patient harm: never events, substandard care, morbidity, mortality

c_wb Risk Factors: severity, frequency, detectability, complexity (1-4 scale)

c_wb handoff & interruptions increase complexity

Better supporting handoffs and managing interruptions is key. See Interruptions, Usability, and Pediatric and Primary Care EMR Workflow on the subject.

Audience Questions/Comments During Technical Feedback on EUP (EHR Usability Protocol)

c_wb Now moving to Breakout sessions: (Red) Tech Feedback on EUP (Green) Building Collaborative Community 4 Improving Usability

I stayed for the Red technical feedback session

c_wb question: will testing occur in typically noisy & distracting environment of real EMR user?

Great question!

“Similar to an under-attack fighter, a busy airport control tower, or a hectic lunchtime restaurant, medical practices can be high cognitive load environments (especially during the flu season in primary care). All four require multitasking and prioritization in the face of interruption and distraction.” (EHR/EMR Workflow System Usability–Roots in Aviation Human Factors)

Going to be very interesting to see how EHR usability, relevant to real-life use of EHRs, will be measured.

c_wb comment: warning that dealing with test system versions, configurations, and seeding realistic patient data will be very difficult

Traditional EMRs are highly customizable (often based on table-driven development). However, they are still not customizable enough, especially when it comes to workflow (which will be necessary to improve EMR workflow from 91 to 14 steps, as mentioned in a presentation). As workflow engines executing user customizable process definitions become more prevalent, EMRs will become even more customizable. So, which set of EMR process definitions will be tested? Since users will likely change these definitions, how do EMR workflows “in vivo” get tested for usability? I suspect we are ultimately looking as more use of participant observation in the wild than summative testing in simulated environments. But then that is more qualitative than quantitative, which makes it difficult to apply to programs for certification of EMR / EHR usability.

c_wb question: are looking where in workflow error occurs? team setting, where in setting?

c_wb Answer: interface is built based on workflow, it is the interface that is tested

What does it mean to say an “interface is built based on workflow”? I think it means that someone analyzes the workflow and then writes software that fits the workflow. The problem is we (developers) aren’t very good at doing this. Is there an alternative? I think there is. Build EMRs with workflow engines executing process definitions so users don’t have to be programmers to change workflow. There are even so-called Design by Doing approaches that allow users to create their own workflows without having to deal with workflow editors. Instead of trying to make sure an EMR fits medical practice workflow before it is installed, give its users the tools to more easily change its workflow. But, then, how do you measure usability? Isn’t usability then really about the tools used to change EMR screens and workflows, not the resulting screens and workflows?

c_wb question: what about variability? comment: reducing usability to number of clicks problematic

Agree: See my comments above…

c_wb audience: need more detailed cognitive model of error besides omission/commission?

Might want to check out the Rouse Human Error Scheme (disclaimer: he was my Industrial Engineering advisor). I’ve not been able to find a relevant paper of his that is not behind a paywall, but a table appears at the end of Improving Human Factors in Marine Maintenance by Clive K. Bright BA, PhD and Simon P. Bell BSc, CEng. (cached)

c_wb audience: would need 2 test 100 users at least for each release, which are frequent

Summative testing does require a test design that averages over a number of similar users performing similar tasks in similar environments (in contrast with current certification scripts that rely on one user (perhaps pretend) performing each task once over the Internet. So presumably more work than current certifications.

Formative and summative usability testing are compared here.

c_wb response: if user interface doesn’t change, don’t need to test, only need 15 test users per category

c_wb response: can still identify critical use errors with smaller groups even if not statistical significant

c_wb audience: different sites can run very different version of same EMR version, test each?

c_wb audience: most important causes of critical error may B in the variability between sites such as how they handle interruptions ehrusability

c_wb audience: 8 week release cycles, could be a lot of testing…

BTW: all of these questioners were thanked and their comments and questions appreciated, that was the purpose of this portion of the workshop

c_wb audience: can eye-specialty vendor adapt usability to their subspecialty workflow?

I recall this audience member commented that in order to obtain EMR certification that they had to add pediatric growth chart to their EHR even though it was of no possible use to any of their customers.

Response: likely should just concentrate those aspects of workflow that are relevant to them [tempted to go back to my Looxcie video to find the exact wording]

c_wb audience: where can we get credible test data? response: building community to provide this

c_wb audience: radiology oncology errors must be reported to state, 1/10000, pool size statistical too small

c_wb audience: why so difficult for gov & vendors to collaborate? response: Question not for this forum (ask ONC)

Believe this question came from a European representative in the audience…

c_wb @kevinmarks re medicine where aviation was in 1940 < coincidence! > conf @ NIST this topic https://twitter.com/c_wb/status/78171827809173505

That it! There are tradeoffs when you tweet instead of write your notes. I’m not a fast typist so I miss stuff. On the other hand there is less transcription from difficult to decipher handwriting. Plus the tweets provide an electronic outline that begs for further electronic annotation. Another thing I like about tweeting notes is that I can retweet other folks in the audience (or sometimes not even in the audience but nonetheless following along) and interact with them.

Cheers!

P.S. Here is that material regarding clinical groupware usability that I promised earlier:

“Great definition…but it just seems so, well, “singleware-ish.” Clinical groupware needs a less abstract definition of usability that is more direct about groupware’s unique usability issues (see the sixth, seventh, eighth, ninth, and tenth quotes from my Clinical Groupware…Key Ideas post). How about:

Clinical groupware usability is…

“The extent to which clinical groupware can be used by specified teams of users to coordinate activity and achieve specified collections of goals with overall effectiveness, efficiency and satisfaction in specified contexts of use.”

“Distributed Cognition takes as its unit of analysis a complex cognitive system: collections of individuals and artifacts that participate in the performance of a task. The external structures exchanged by agents of complex cognitive systems comprise its “mental” state and unlike individual cognition, where mental states are inaccessible, these states are observable and available for direct analysis.”

“The Human Factors in Computing community has a…challenge [to] find ways to test and evaluate technological impacts on groups. It’s difficult enough to get meaningful results that take into account differences in experience and individual differences of users to their reactions to user interfaces. But at least it’s possible to get volunteers to sit down with word processing systems and spreadsheet programs for relatively self-contained tasks. It is more difficult to “stage” a realistic group-work setting in a lab and have volunteers use the system in a way that provides meaningful data. Methodologies for testing individual user interfaces don’t apply as well to group support systems. As a result, CSCW [Computer-Supported Cooperative Work] is looking more to anthropology to find methodologies for studying groups at work in their natural settings.”

“Until recently, most user interface research has focused on single-user systems. Groupware challenges researchers to broaden this perspective, to address the issues of human-user interaction with the context of multiuser or *group* interfaces. Since these interfaces are sensitive to such factors as group dynamics and organizational structure—factors not normally considered relevant to user interface design—it is vital that social scientists and end users play a role in the development of group interfaces.”

“Evaluating groupware ‘in the field’ is remarkably complex because of the number of people to observe at each site, the wide variability of group composition, and the range of environmental factors that play roles in determining acceptance”

“Five factors contributing to groupware failure…:

1. Groupware applications often fail because they require that some people do additional work, and those people are not the ones who perceive a direct benefit from the use of the applications.
2. Groupware may lead to activity that violates social taboos, threatens existing political structures, or otherwise demotivates users who are crucial to its success.
3. Groupware may fail if it does not allow for a wide range of exception handling and improvisation that characterizes much group activity.
4. We fail to learn from experience because these complex applications introduce insurmountable obstacles to meaningful, generalizable analysis and evaluation.
5. The groupware development process fails because our intuitions are especially poor for multiuser applications.”

Follow me on Twitter at @c_wb.

Check out the EHR.BZ REPORT on EMR / EHR Workflow, Usability, Safety and Productivity!

I’ve been using EHR.BZ as an EHR-themed URL shortener for tweeting about EMR / EHR workflow, usability and productivity from my @c_wb Twitter account. I decided to use EHR.BZ’s home page to archive select links about these topics. After looking at the usual free open source content management systems (Wordpress, Joomla! and Drupal) I decided to create my own. Since by some accounts the DRUDGE REPORT is one of the most usable websites around, I borrowed some ideas, wrote some PHP, and the EHR.BZ REPORT on EMR / EHR Workflow, Usability and Productivity resulted.

…

Got some other ideas about what I might do, so check back! Or forget about the EHR.BZ REPORT on EMR / EHR Workflow, Usability, Safety and Productivity and just follow me on Twitter: @c_wb.

I haven’t posted here for a while. I’ve got a lots of ideas for future posts, plus drafts I need to get around to polishing and publishing. In the mean time, Twitter sure is fun and I’ve met some great folks there with similar interests.

Ironic that I started tweeting to get attract traffic to the blog (“A Twitter Holiday: Sun, Sand, Surf, Smartphones, Short URLs, Social MEdia, and EMR Workflow Systems”) but was myself attracted from my blog to Twitter. Perhaps the pendulum will eventually swing back.

In the meantime…

I tweet about basically the same stuff (plus a dumb joke once in a while). I hope you’ll follow me (mention this blog and I’ll be sure to follow you back). Or, if you don’t have an account, head over to Twitter and get one.

Cheers

Chuck

http://twitter.com/chuckwebster

@chuckwebster