Don’t Curate the Data
It’s tempting when we talk to others about our ideas to only want to share the good stuff. To only share the things we think are logical, sound reasonable, maybe only the things we think (or hope) will make us seem smart and focused. But this tendency to re-frame our real experiences and distill them into nice little stories we can tell people over coffee or a beer can be a dangerous setback to getting better at a new skill set.
Trying Too Hard to Look Good
Why? Because sometimes we are so busy trying to think about how to tell (or should I say sell) others on what we’re doing or thinking that we scrub our memories clean of the actual messy chain of events that led us to come up with the polished version. That messy chain, and every twist, turn, and chink in its construction, is the raw knowledge from which we can learn about how we, or others, actually accomplish things. I’ll call it “the data.”
So this fear of how others will perceive our process is one thing that gets in the way of having good data about our process. We start to curate the data to make ourselves more acceptable to others.
But we need this data to gain a meaningful awareness of what we actually do to produce a certain outcome. This is even more important when we try to figure out how to reproduce a mental outcome.
Maybe you came up with a winning idea once, but now you’re not sure how to get the magic back. Or maybe you want to pass your strategy on to a younger colleague or friend, but don’t really know what you did. Maybe you’re hoping to learn from someone else who succeed at thinking up a breakthrough solution, but they say “I really don’t remember what I did. It just sort of came together.”
Which brings us to a second thing that works against having access to good data about our own interior processes and patterns. Memory.
Mining Memory is a Tricky Business
We all know we don’t have good memories, even when we are trying hard (studying for tests in school, or trying to remember the name of every person in a group of ten new people you just met are classic examples). Memory is imperfect (we have weird, uncontrollable gaps in what we retain). Memory is selective (we have a tendency to be really good at remembering what happened during highly emotional events, but not during more mundane or routine moments). Memory is pliable (the more we tell and retell a version of something that happened to us, the more likely we are to lose the actual memory in place of our story version).
These tricks of memory not only frustrate us when we try to observe and learn from ourselves, but also when we try to learn from others.
There have been lots of interviews with famous scientists who made discoveries asking them about how they did it. But their self-reported stories are notoriously unreliable or have big gaps because they, like us, are subject to the fickle whims of memory and the hazards of trying to tell your own biography one too many times. Mining memory for useful insights is a tricky business.
So memory and lack of awareness (or mindlessness) cause us to lose access to the precious data we need to be able to see our behaviors and patterns from a larger perspective in order to learn from them and share them.
When I first started learning about scientific discovery, recognizing these pitfalls of bad memory and mindlessness caused me a lot of annoyance. I would think of a great example of a scientific discovery, such as a discovery that shared similarities with an area or question I wanted to make discoveries in. I’d think, “Perfect! I’ll go read up on how they did it, how they discovered it. What were they reading, what were they doing, who were they talking to?” But of course, answers to those questions wouldn’t exist!
Maybe the discovery was of limited interest so nobody bothered to ask those questions and now the discoverer had passed away. Or maybe the discovery was huge and world changing but the histories told about it tended to re-hash the same packaged myths—like Newton and the apple falling inspiring ideas about gravity, or Einstein taking apart watches from an early age leading to picturing little clocks when working out the effects on time of traveling near light speed in special relativity. Part fact, part fiction, these stories leave hundreds of hours of more mundane moments, links in the mental chain, unilluminated. Good data that could guide future generations gets lost, sacrificed on the altar of telling a whimsical story.
So when I sat down in September of 2018 to start trying to work out a more modern definition of scientific discovery—something pragmatic that you could use to figure out what to do during all those mundane moments—I kept thinking about how to better capture that process of obtaining insights, as you go.
That’s when I realized we already have the methods the problem is we always want to curate the story told after the fact. And rather than curating the data that make it into the story (i.e., creating an executive summary and redacting some things), we end up actually curating the source data itself (i.e., never gathering the evidence in the first place). In other words, rather than just leaving out parts of the story, we actually tune out to parts of the story as we are living it, so that we literally lose the memory of what happened all together.
But that story is the raw data that fields like metascience and the “science of science” need to help figure out how scientists can do what they do, only better. And as scientists we should always be the expert on our own individual scientific processes. The best way to do that is to start capturing the data about how you actually move through the research process, especially during conceptual and thinking phases. Capture the data, don’t curate the data.
A Series of Events
Let me give you a real life example to illustrate. As I said, I sat down to try to come up with a new definition of scientific discovery. I’m a physicist by training. Defining concepts is more a philosopher’s job, so at first I had a hard time taking myself and any ideas I had seriously. I got nowhere for three months; no new ideas other than what I had already read. Then one day a series of events started that went like this:
I read a philosophy paper defining scientific discovery that made me very unhappy. It was so different than my expectation of what a good and useful definition would be that I was grumpy. I got frustrated and set the whole thing aside. I questioned why I was studying the topic at all. Maybe I should stick to my calling and passion, physics. I read when I’m grumpy, in order to get happy. So I searched Amazon. I came across a book by Cal Newport called So Good They Can’t Ignore You. It argued that passion is a bad reason to pursue a career path, which made me even grumpier; so grumpy I had to buy the book in order to be able to read it and prove to myself just how rightfully disgruntled I was with the premise.
Newport stresses the idea of “craftsmanship” throughout his book. I was (and still am) annoyed by the book’s premise and not sold on its arguments, but “craftsmanship” is a pretty word. That resonated with me. I wanted to feel a sense of craftsmanship about the definition of scientific discovery I was creating and about the act of scientific discovery itself.
I didn’t want to read anymore after Newport. So I switched to watching Netflix. By random chance I had watched a Marie Kondo tidying reality series on Netflix. Soon after, Netflix’s algorithm popped up a suggestion for another reality series called “Abstract: The Art of Design.” It was a series of episodes with designers in different fields, like architects, Nike shoe designers, theater and popstar stage shows set designers, etc. It pitched the series as a behind the scenes look at how masters plied their craft. Aha, craftsmanship again! What coincidence. I was all over it (this was binge watching for research, not boredom, I told myself). I was particularly captivated by one episode about a German graphic designer, Christoph Niemann, who played with Legos, and whose work has graced the cover of The New Yorker more than almost any other artist. The episode mentioned a documentary called “Jiro Dreams of Sushi.”
Stick with me. Do you see where this is going yet? Good, neither did I at the time.
So I hopped over to Amazon Prime Video to rent “Jiro Dreams of Sushi” about a Japanese Michelin rated chef and his lifelong obsessive, perfectionist, work ethic regarding the craft of sushi. At one point the documentary showed a clip of Jiro being named for his Michelin star and they mentioned what the stars represent: quality, consistency, and originality. Lightbulb moment! Something about the ring of three words that summed up a seemingly undefinable craft (the art of creating delicious food) felt like exactly the template I needed to define the seemingly undefinable art of creating new knowledge about the natural world.
So I started trying to come up with three words that summed up “scientific discovery”. Words that a craftsman could use to focus on elements and techniques designed to improve their discovery craft ability. There were more seemingly mundane and off-tangent moments over a few more months before I came up with the core three keywords that are the basis of the definition I am writing up in a paper now.
The definition is highly unique, with each term getting its own clear sub-definition that helps lay out a way to critically examine a piece of research and evaluate it for its “discovery-ness”, i.e., its discovery potential or significance. It’s also possible to quantify the definition in order to try and rank research ideas relative to one another for their discovery level (minor to major discovery).
It’s a lot better idea than some of the lame generic phrases that I came up with in the early days, like “scientific discovery is solving an unrecognized problem ” (*groan*).
On an unrelated track at that time, I was reading Susan Hubbuch’s book, Writing Research Papers Across the Curriculum, and had come across her idea that you create a good written thesis statement by writing out the statement in one sentence and then defining each keyword in your statement using the prompt “By <keyword> I mean…”. So then I took the three keywords I had come up with and started drafting (dare I say crafting?) their definitions in order to clarify my new conception of “what is scientific discovery?”
So that’s the flow…my chain of discovery data:
Reading an academic paper led to disgust; disgust led to impulse spending; impulse spending brought in a book that planted the idea of craftsmanship; craftsmanship led to binge-watching; binge-watching led to hearing a nice definition of something unrelated; the nice definition inspired a template for how to define things; and simultaneously reading a textbook suggested how to tweak the template to get a unique working definition down on paper.
How do I know all this? I wrote it down! On scraps of paper, on sticky notes, in spiral notebooks, in Moleskines, in Google Keep lists, Evernote notes, and One Note notes (I was going through an indecisive phase about what capture methods to use for ideas).
I learned to not just write down random thoughts, but also to jot down what inspired the thought, i.e., what was I doing at the moment the thought struck—reading something, watching something, eating something, sitting somewhere, half-heartedly listening to someone over the phone…(Sorry, Mom!)? Those are realistic data points about my own insight process that I can use later to learn better ways to trigger ideas. (And, no, my new strategy is not just to watch more Netflix.)
Make a Much Grander Palace of Knowledge
Instead of trying to leave those messy, mundane, and seemingly random instigators out, I made them part of my research documentation and noted them the way a chemist would note concentrations and temperatures, a physicist energies and momenta, a sociologist ages and regions.
And then I promised myself I wouldn’t curate the data. I wouldn’t judge whether or not impulse book buying is a great way to get back on track with a research idea, or whether or not Marie Kondo greeting people’s homes with a cute little ritual is a logical method of arriving at a template to devise operational definitions. I wouldn’t drop those moments from memory, or my records of the research, in order to try and polish the story of how the research happened. I’ll just note it all down. Keep it to review. And maybe share it with others (mission accomplished).
Don’t curate the data, just capture the data. Curation is best left to analysis, interpretation, and drawing conclusions, which require us to make choices—to highlight some data and ignore other data, to create links between some data and break connections among other data. But think how much richer the world will be if we stop trying to just tell stories with the data we take and start sharing stories about how the data came to be. The museum of knowledge will become a much grander palace. And we might better appreciate the reality of what it is like to whole-heartedly live life as a discoverer.
Related Content on The Insightful Scientist
Blog Posts:
How To Posts:
- —
Research Spotlight Posts:
- —
How to cite this post in a reference list:
Bernadette K. Cogswell, “Don’t Curate the Data”, The Insightful Scientist Blog, August 2, 2019, https://insightfulscientist.com/blog/2019/dont-curate-the-data.
[Page Feature Photo: The gold dome in the Real Alcazar, the oldest used palace in Europe, located in Seville, Spain. Photo by Akshay Nanavati on Unsplash.]