Emergency Department sense-making series: how perceptions on information influence the actions taken.

Note This blog forms part of the sense-making in the Emergency Department series. If you are unfamiliar with sense-making and SenseMaker®, please read the previous blogs in the series, as it builds on each other. (Else, the blogs will be incredibly repetitive). The series shares research funding conducted to understand sense-making in the ED better. Briefly on the method: participants were asked to tell a short descriptive story that was captured in the SenseMaker® tool. After telling their story, the participants self-interpreted their story within a specialised framework, the results of which provided meta data to explore for patterns of the sense-making dynamics. This blog delves deeper into results about perceptions on information.

Background

In the workplace, the flow of organizational happenings provides information on a constant basis. Regarding information, sense-making theories speculate about the kind of processes that occur when people tap into the information surrounding them. The information may enter their consciousness via the senses, language, etc. and they may actively seek information.

An early phase of sense-making is noticing what to respond to – identifying happenings or sensory perceptions as noteworthy and transforming it into words – thus attaching meaning. Noteworthy information depends on a variety of factors including experience, priming, identity, agency, context, mental frame (to name a few).

Before noticing an anomaly, there needs to be a flow of information. And even though it is commonly assumed that more information leads to better sense-making and thus decision-making this is not true. More information improves performance (sense-making) to a point, but after a certain point, additional information isn’t helpful, in fact it could hamper performance. Thus, in the survey we wanted to determine if people have the information capacity for sense-making. We asked if storytellers felt that in their story they had enough information, too little information or if they were overwhelmed.

Research Analysis

The self-analysed stories provided quantitative data, that we explored patterns. Using this tool, individual responses provide limited insights, instead the analysis is focused on finding patterns.

We purposefully designed the dyad using extremes, at the one extreme information was completely absent whereas at the other it was present in excess i.e. swamped. Storytellers answered the question by moving the blue dot (Figure 1), with the optimal state being somewhere in-between the two extremes rather than at an extreme. The findings (Figure 1) suggest most participants felt there was enough information, leaning towards feeling the information are too much, too early, swamped.

Availability of information

Figure 1: Availability of information

Sense-making is about making enough sense to act in the world. After determining that there is enough (often too much) information available, we asked what storytellers did when the selected or noticed information did not match the possible explanations.

We asked the question in a triangular grid with labels at each corner. Storytellers moved the blue dot anywhere inside the triangle to show how the variables (labels at each corner) trade off against each other. Should the dot be in the middle of the triangle, it indicates all three variables are equally represented in the story told (Figure 2).

The labels/variables were based on Klein’s data/frame model for decision-making . That his team developed with the US military to explore what military decision-makers do when information and explanation does not match. Klein et al., argued that a key aspect of sense-making matching data (information) with a mental frame and/or adjusting a mental frame around the data.

The data/frame model asserts that data are used to construct a frame (story, script, schema) that explain the data and guides the search for additional data. Concurrently, the frame determines what data or additional data if needed are sought.

There are six building blocks in the data/frame model: elaborating an existing frame, questioning a frame, preserving a frame by explaining anomalies away, comparing alternate frames and reframing to replace the existing frame. The nature and outcome of the sense-making differs depending on whether a person is elaborating or questioning a frame, or simply accepting it.

We asked storytellers if in their story when information and explanation did not match whether they accepted that is not matching i.e. live with the difference; rechecked information; or sought additional information.  

The findings (Figure 2) seem well distributed with some storytellers accepting the mismatch between information and data, others wanting more information and re-checking the data etc.

Figure 2: Actions when information and explanation do not match.

It became interesting when we searched for patterns (Figure 3). We explored the actions against those who felt overwhelmed with information with those that felt the information lacking. And found that those who felt information was lacking, were less likely to accept the anomaly, whereas those who felt swamped were more likely to accept the anomaly without questioning or finding more information.

Plausible reasons for accepting a mismatch between the information and explanation (preserving the mental frame) includes not realizing the signals of a potential problem, or perhaps storytellers felt it’s not their job to deal with the discrepancy. It’s been shown that when people can’t do something with information, they’re less apt to notice emergent patterns.

Whatever the reason, in dynamic environments it is important to notice ‘weak signals’ and emergence. Even though accepting anomalies could be useful (i.e. reduce the amount of incoming ‘noise’) it could also be detrimental resulting in operational or clinical failure. This point requires a deeper delve to understand the types of information that was ignored/accepted.

Figure 3: Perceptions about availability of information and actions taken when available information and explanation did not match.

Emotion

Processing information is not only about the cognition, it also extends to emotional framings of knowing. Literature about the role of emotion during sense-making theorize that when a situation or workplace is experienced negatively, it impacts on the individuals’ ability to express themselves, their willingness to share information and listen to others. We wanted to see if we could find patterns in how storytellers felt about their story, their actions, and views on information.

Storytellers were asked to rate how they felt about their story on a 5-point Likert scale ranging from strongly negative to strongly positive. We used this question to ‘slice’ the findings on information and it emerged that those who rated their emotions as strongly negative, negative, or neutral were more likely to accept a mismatch between data and explanation.

Figure 4: Actions taken when information and explanation did not match and retrospective feeling about the story shared.

To summarise, we considered if there is enough information to make sense, and storytellers said that there is enough information, often too much. We then explored what people do when what they expected to find and what they found did not correlate, and overall the responses seemed well-distributed, until we started pattern-seeking. In this blog I discussed three filters used for pattern-seeking – views on information, actions taken when there is a mismatch, and retrospective feelings about the stories.

Why does this matter?

We often hear about connecting dots; however, it is not always about connecting dots or the amount of incoming information. Rather, it’s about knowing which dots to connect and how to patch fragments of information together. How people make sense impacts their decisions, actions and what happens next. This can be improved by targeted training.

The underlying values of the system should also be questioned. The anthropologist Mary Douglas argued that organisations systematically direct perceptions into ways that are compatible with the authorized relationships. Thus, the structure of the organization is partially responsible for what will be noticed and responded to.

Training to improve information-processing and sense-making.

Cognitive Skill Development

The Shadowbox Training method involves scenario-based training that allows people to see situations through the eye of experts without the experts being present. This could help newcomers or novices to know how to piece information together, and appropriate actions.

Another method, the Premortem method of Risk Assessment is imagining a proposed plan has failed, as starting point, and then to work back to the current situation to identify the types of information that should not be ignored.

Participatory Workshop: using narratives to delve deeper into the patterns.

The storytellers are the best situated to delve deeper and explore the meaning of the findings and subsequent actions. SenseMaker® can be used to monitor and evaluate subsequent actions.

Research

Cognitive task analysis aims to understand tasks requiring a lot of cognitive activity for example decision-making, problem-solving, memory, and judgement. It can be used to examine performance differences between novices and experts, mental workload, decision-making, and to understand mental models. Using CTA reasoning can be unpacked, and tacit knowledge made explicit. This is important in dynamic environments.  Another way is to interview people after critical incidents to understand the cognitive and information recognition pathways.

These research methods are pre-hypothesis forming, meaning it may uncover hypothesis for further research in decision-making and sense-making.