Seeing is believing: new ways to understand search and discovery

Have you ever had that moment when you’ve seen something out of the corner of your eye, then turned to look but it’s gone? We’re left feeling cheated, as if some magic took place that was never intended for our eyes. But the reality is often more prosaic: cells in the human retina are arranged such that movement and contrast are better perceived around the peripherae, while the central region is better suited to colour and detail. It’s a simple explanation, but one that reminds us that in order to better understand, we sometimes need to see in different ways.

I’ve often observed that searching for information presents a similar challenge: in order to satisfy complex information needs, we must articulate those needs faithfully and then perceive their effect in the response from the environment. We become partners in an exchange: a dialog between user and search system that can be every bit as rich as human conversation. Crucially, the better we can articulate our own needs, the more trust we can place in the response.

Nowhere is this truer than for structured searching, where the goals of accuracy, transparency and reproducibility are at their most acute. But this is precisely where existing formalisms fall short. Just when we most need an effective way of seeing, we are left with words, lines and numbers. Instead of using perception to understand the structure of our information needs, we are forced to rely on cognition, with its associated human costs of effort and error.  Instead of using approaches that allow us to think fast, we rely on formalisms that force us to think slow. Does it have to be this way? In this post, we’ll examine alternative visions for search and discovery.

Let’s start with a familiar example: a complex search on the subject of ‘Galactomannan detection for invasive aspergillosis in immunocompromised patients’. In its traditional form, this would be articulated via a form-based query builder as a series of interconnected Boolean expressions:

1 "Aspergillus"[MeSH]

2 "Aspergillosis"[MeSH]

3 "Pulmonary Aspergillosis"[MeSH]

4 aspergill*[tiab]

5 fungal infection[tw]

6 (invasive[tiab] AND fungal[tiab])

7 1 OR 2 OR 3 OR 4 OR 5 OR 6

8 "Serology"[MeSH]

9 Serology"[MeSH]

10 (serology[tiab] OR serodiagnosis[tiab] OR serologic[tiab]) 11 8 OR 9 OR 10

12 "Immunoassay"[MeSH]

13 (immunoassay[tiab] OR immunoassays[tiab])

14 (immuno assay[tiab] OR immuno assays[tiab])

15 (ELISA[tiab] OR ELISAs[tiab] OR EIA[tiab] OR EIAs[tiab])

16 immunosorbent[tiab]

17 12 OR 13 OR 14 OR 15 OR 16

18 Platelia[tw]

19 "Mannans"[MeSH]

20 galactomannan[tw]

21 18 OR 19 OR 20

22 11 OR 17 OR 21

23 7 AND 22

 Each line consists of a series of keywords, operators and controlled vocabulary terms, which are connected via logical operators and Boolean expressions. The glue that binds all this together is the line numbering (a mechanism not entirely dissimilar to that used in early programming languages such as Unstructured BASIC). 

Now, here is the test. If you were asked to describe how this search is structured, what would you say? How many sub expressions does it contain? How are they connected?

Clearly all these questions are answerable, and more so to the trained eye. But the point is that the answers are not directly visible. Instead, we must proceed through a sequence of steps. We must first retrieve from memory a program for interpreting line-by-line searches, and then implement it. In doing so we must hold data in short-term memory, and keep track of where we are and where we are going, while holding on to any intermediate results. The process is mental work: deliberate, effortful, and orderly—a prototype of slow thinking.

Now let’s examine three alternative visions, which are motivated by the principle of migrating thinking from slow to fast; from cognition to perception. We’ll start with what we call the Nested View. This view and those that follow can all be invoked by opening traditional, text-based searches in 2Dsearch.

Nested View

We’ve introduced this view before, so will review it only briefly here. In short, it provides a view which maps hierarchical structure onto a series of nested containers. The benefit is that the grouping and containment become immediately apparent:

In our example, we see that the search consists of a conjunction of two disjunctions (Lines 7 and 22), the first of which articulates variations on the fungal infection concept, while the latter contains various nested disjunctions to capture the diagnostic test (serology) and associated procedures. Moreover, by displaying them as nested groups with transparent structure, it offers support for abstraction, whereby lower-level details can be hidden on demand by the user. In addition, it is now possible to give meaningful names to subgroups, so that they can be saved and reused as modular components.

However, the Nested View has its drawbacks. Although it provides a fine degree of control over the shape and layout of the groups, the fact that they are rendered as blocks with operators in their headers isn’t for everyone.  Let’s examine an alternative.

Tree view

Another way to understand the hierarchy embodied in complex searches is to apply a metaphor that is almost universally understood: the family tree. In this view, the search is represented as a visual hierarchy, with the root node (Line 23 in this example) at the top, and each level below represented as successive generations of children:

In this example, we have displayed the search in its entirety, and shrunk it to fit across the page. But it’s easy enough to zoom in and out, and reveal just the higher levels:

Or to close branches on demand, and focus on one particular region of the tree:

Like the Nested View, the Tree view also maps conceptual hierarchy onto physical hierarchy, but in a manner that emphasises branching over containment. But is hierarchy the most important aspect of a search? With that in mind let’s examine a third view.

Inline view

The use of Boolean strings to represent complex searches may be inefficient and error-prone, but it does offer one key benefit: the ability to be read in a left to right manner. Of course, this attribute may reflect nothing more than the inertia of decades of convention, but there remains something useful about being able to read searches as a series of statements or commands. Is it possible to support this principle in a visualisation? This is where the Inline View comes in. 

Like the Nested View, the Inline View maps conceptual hierarchy onto physical containment, but this time in a manner that aligns groups along a common midline, giving rise to a natural left-to-right reading:

Notice that in this view we elevate the operators to same level as content items so that they appear in sequence within the left to right reading. This means that we can also reduce some of the chrome around groups, leading to a ‘cleaner’ layout. Again, we’ve shrunk the above image to fit across a single page. But as before it’s trivially easy to zoom in and out, e.g. to reveal just the higher levels:

Or to close branches on demand, and focus on one particular region of the search:

Of course, the Inline View has its strengths and weaknesses too. Moreover, it’s important to recognise that while this article focuses on new ways of seeing, the real benefit is in the interaction: to modify a search, you can simply move terms from one block to another, or create new groups by combining terms. You can also cut, copy, delete, and lasso multiple objects. If you want to understand the effect of one block in isolation, you can execute it individually. Conversely, if you want to exclude one element, you can temporarily disable it.

In closing

In this post we’ve explored three different ways to visualise complex searches. In each case, we’ve tried to show that it is possible to represent complex logic in a manner that supports thinking both fast and slow. In future posts, we’ll explore how to use these views to debug and optimise complex searches.

Evidently, each view has its own strengths and weaknesses. Indeed, none of them is a silver bullet: the point is that they all reveal different aspects of a search, and offer different insights and ways to understand them. It is through their collective diversity and flexibility that we discover new ways of seeing.

A picture is indeed worth a thousand words. To see for yourself, visit 2Dsearch, and let us know what you think.