Data Sonification and Life Forms

One of the things we do as AS IF Center is to catalyze collaborations through Art-Science Matchmaking. I hope you enjoy this guest post by Judith Casseday about one such collaboration with Rob Dunn Lab.      – nl

I have a keen interest in data sonification that furthers our understanding of the data. This
blog post by Mark Ballaro and George Smoot increased my interest in exploring how modal/timbral shifts that are set in a familiar, well-tempered scale spectrum might illustrate data-driven relationships. Recently, I read a notice from AS IF about collaborating with Rob Dunn’s Lab on a project studying microbiology of sourdough. It felt like a dream! I have a two and half year old sourdough starter which is used to create 75% of the bread I eat, I recently studied cell biology and neurobiology, I have a deep interest in molecular chemistry about which I am just learning, and I am looking for a data sonification project. I sent the Dunn Lab an inquiry, they checked out my sound work, and I was invited to participate.

First I met with the sourdough folks at Rob Dunn’s Lab — Erin McKenney, post-Doctoral Fellow in Microbiome Research and Education and a research lead on the sourdough project, and Lauren Nichols, Dunn Lab Manager. I learned that the sourdough project is looking at the ecology of sourdough starter communities as relates to yeast and bacteria growth in flour when exposed to water and the local microbial environment. I attended a Dunn lab staff meeting and learned about the amazing research they are doing. All the projects are basically looking at how the smallest phenomena impact much larger phenomena and vice versa, the micro to macro to micro feedback loop. They keep finding that diversity is the key to sustainable growth and a healthy environment. I left the meeting excited and inspired! Next stop will be the AS IF Center in October, for a retreat with some of Rob Dunn’s collaborators on the sourdough project.

I wanted to sonify some data to prepare for the sourdough meeting, so I reached out to the Dunn Lab folks, and Erin McKenney sent me a data set to try my hand at. The data, from Erin’s dissertation study, is about nine lemur babies belonging to three different species, and enumerates how the microbial colonies in their guts evolve from birth to weaned. We have identifiable parameters that can be orchestrated to show changes over time. Perfect!

The lemur microbial data is on a massive spreadsheet with lots of terminology I don’t know… yet. This will be an interesting process as we work out exactly what the sonic illustration will depict. I sense that certain data will lend itself to sonification and that is the part I do not yet know. After spending some time studying the spreadsheet, I asked Erin how we can cluster some of the microbial data together, and she sent me the data sheets for the bacteria, classified at the phylum and class levels. The phylum-level data became my focus as there were only 35 phyla as opposed to 95 classes and 255 strains of bacteria. One of the lemur mothers had triplets so I decided to put together phylum-level profiles on this small group. Culling through the data for these specific individuals narrowed the number of phyla down to 24, then I made an arbitrary cutoff point of >.00 density for each phylum (Erin said this was fine and is actually a tool scientists use to declutter data). Now I was down to 15 phyla – a manageable number for timbral illumination.

The microbes were collected from the three lemur babies at six time points, from birth to nine months old. These time points were birth, nursing, introduction of solid foods, regular consumption of solid foods, and two times as they were weaning. Microbes were collected from the mother when she gave birth. Erin had the brilliant idea to have the mother’s phylum-level profile (which represents the stable adult community, and does not change over time) be a drone under the babies’ phylum-level profiles in the sound illustration. This allows you to hear when the profiles diverge and when they converge.

The sonic substance for all this is a phyla megachord that stretches from G1 to G5. Each phylum is voiced by a single pitch, so, for example, Protobacteria is G1. Since there are only thirteen pitches in a chromatic scale, some of the phyla would land on the same pitch, different octaves. There were five phyla that tended to have the highest presence in each sample, so I made them the Gs, and all the rest had separate, distinct pitches. I used amplitude to render the amount each phylum was present in each sample.

The next question was how to voice the individual profiles in order to hear the data as clearly as possible. After much experimentation, I decided to represent the mother’s voice as a woodwind with steady, slightly pulsing tone throughout. I chose bell-like voices for the three lemur baby profiles, letting each phase ring out four times over the mother’s profile. The idea is to listen and compare the mother’s profile with the babies’ profiles. Listen for the change (or lack of change) as the each stage rings in four times. What you hear is a uniformity of tone at birth that becomes more dense and dissonant as the phyla diversify with the babies’ diversifying diet. Then the final wean profile settles into more consonance with the mother’s profile.

You can listen to the lemur data sonification project here, and soon I’ll begin experimenting on sonifying data from the sourdough project.

-Judith Casseday


Reflections on Art as a Pathway to Effecting Change

This is a guest post from AS IF Center resident Cynthia Reeves, who was the Art of the Climate resident in March.

The biggest takeaway from the first annual ClimateCon held in Asheville March 19-21 was how climate change affects every facet of society—from major considerations such as protecting the global food supply in the face of increasing threats caused by climate change to more minor considerations such as how climate affects day-to-day consumer choices. The conference attendees represented every sector of society—business, government, NGOs, non-profits, the arts—all responding proactively to climate change. I was especially impressed by how many corporations have established sustainability departments not only to burnish their reputations as good actors on the global stage but also to bolster their bottom lines. Participants effectively dispelled the myth that mitigating climate change impacts increased costs. The opposite is true.

The conference, hosted by The Collider, has as its mission to deliver a “collaborative experience with a wide variety of business and science professionals who come together to advance the development of data-driven products and services.” Thus, it wasn’t surprising that the arts were not a significant presence. As a writer whose current work incorporates the science of climate change, networking with scientists, business people, innovators, public sector sustainability professionals, academics, and NOAA and NCEI data managers did provide me with a wealth of information to use in my fiction. Scientific facts alone, however, aren’t enough to convince climate change skeptics; art can be used to communicate on a human scale how climate change affects us all. Translating those facts into “story” and characters grappling with a world in which climate change exists may help motivate a skeptic’s change in thinking.

Cynthia Reeves writing upon landing in the Arctic Circle

Likewise, during my residency at AS IF, I had the opportunity to reflect on art’s role in providing a pathway to effecting change by presenting a talk—“Of Ice Floes, Whale Bones, and Abandoned Mines”—about my experiences aboard the tall ship Antigua as a participant in the 2017 Summer Solstice Expedition to Svalbard (Norway). The Arctic Circle sponsors this artist-scientist collaborative residency. The talk focused on the otherworldly beauty of the Svalbard archipelago and how it has inspired my fiction. As part of that presentation, I had the privilege of screening “Moving Image Study of Smeerenburg Glacier,” a six-minute film of a massive glacier calving set against the quiet, sculptural splendor of an ice field. The British cinematographer and filmmaker Adam Laity—one of my fellow Arctic travelers—created the film from footage and sound captured by him and others as we witnessed this spectacular event.

The talk was a study in juxtaposition: the Arctic’s pristine natural landscape contrasted with the human imprint upon this fragile environment. I wanted to emphasize the positive: that we can preserve this environment. The talk reflected that positivity—but also the inherent contradictions—of life in the Arctic now. At the very least, I hoped for the audience to be awestruck by nature and (possibly) nature’s ability to heal itself. Given the audience’s reaction as the film unfolded—the gasps and wonderment and even tears it provoked—I believe the talk succeeded.

-Cynthia Reeves

The art-science conversation: two resources for deep reflection

What do we mean when we say “art-science?”

Should we encourage more of it? Can the practice of art-science, or “STEAM,” improve our schools? Since art and science have different goals and different approaches, how should we evaluate art-science? How can we level the playing field between art and science, giving each equal respect and opportunities for support? Why don’t the contributions of art to science mirror the contributions of science to art? Can we change that? Should we?

There is a growing recognition that artists and scientists alike seek truths in the world, explore new territory, build on past work, observe with the senses, generate new tools and techniques, solve problems, often collaborate, and use creativity. In recent years, many new initiatives have emerged to re-integrate art with science. However, there are probably as many ways to define art-science as there are practitioners of it. So how do we talk to each other about what we’re doing?

Two recent symposia, both available online, offer opportunities for deep engagement in this conversation. Below are links to the recorded discussions. There’s a lot of rich material worth exploring, and I hope you reflect on them and leave your comments here.

I will close with a challenge:  Note that a lot of us elephants in the room are white… what are we going to do about that?

Strange Attractors: Art, Science, and the Question of Convergence


Art & Science: The Two Cultures Converging 

Why does art-science matter? Tell funders.

Over the past few years I’ve been doing some grant searches for funding opportunities that bring the arts and sciences together in some way. Grant programs focused on art-science are fairly sporadic. For well-established projects, the National Science Foundation’s Advancing Informal Science Learning (AISL) can be a fit, but NSF grants are notoriously competitive, awarding about 10% of applicants. Alfred P. Sloan Foundation funds a few projects through its Public Understanding of Science. The Wellcome Trust has a great track record of funding arts projects that deal with biomedical topics, for artists who live and work in the UK or Republic of Ireland.

Doing a grant search on the Foundation Center Online, using the controlled-vocabulary search terms, there is no entry for art-science. Writing in “art-science” as a keyword turns up a modest number of grants nationwide, many of which are “state of the art” science technology, or general education grants for “arts and sciences.” “STEAM” yields even fewer results, often for things like “restoration of steam engine,” etc. Once you further narrow the searches geographically (to make them most useful for actual grant writing) the field is pretty limited.

from little acorns...

from little acorns…

Philanthropists want to make a difference. Funders want their giving to have the biggest impact it possibly can, effecting change in the areas they care about. What can we do to convince funders that art-science matters? How can we change the funding landscape and increase art-science opportunities?

Here are three ways to start making change in the philanthropy environment.  Please add your own and share!

One. Reflect: As practitioners of art-science – whether it’s using science materials and methods to make cutting-edge art, using arts to communicate science narratives, blending both art and science to explore in an open-ended manner, using art to teach science in or out of school, or whatever you do that brings art and science together – ask yourself, why does art-science matter?

Two. Communicate: Each time we convene in a think tank, seminar, symposium, workshop, etc. let’s invite funders as well. We know art-science makes interesting work. We know that our culture’s habit of thinking of art and science as separate  has not served us well over the past few centuries, and that reuniting them is important. We know that art-science is nearing a tipping point, with myriad projects germinating in every discipline, in many regions of the country and internationally, in formal education and outside of it. How can we communicate that excitement to funders? With education in crisis, how can we help funders recognize that interdisciplinary art-science is an effective way to re-ignite curiosity, to help learners think of themselves as investigators, as makers, as fully human, with voices worth hearing?  At every art-science event, small and large, let’s invite funders to the table. Ask for their input, give them a role in this movement.

Three. Measure: With every art-science project, keep track of its outcomes. It’s a pain, but it can help you think about what you are aiming to do, then track how well you did it. If you go for funding, tuck a little bit into each budget for evaluating outcomes, even if you don’t have to. It shows you mean business, and it will help us as a movement to make a better argument for increased funding.

In talking to neighbors, friends, and coworkers about what I do in art-science, I’ve been so surprised lately at the rapid growth of this movement. A decade ago, mentions of my art-science collaborations were met with cocked head, furrowed brow, and queries of “How would you bring those two together? They seem so diametrically opposed!” Now, when I talk about art-science, it’s more often met with “Oh, yeah, I went to this cool poetry reading about physics last week” or “Do you know about _____ (insert name of art-science person I’ve never heard of, whom I then tweet next day on my #artscience du jour).” The fact that art-science is becoming not only well understood but even familiar to the general public is a sure sign that we are building a movement.

Getting on board with funders takes more time. The philanthropic landscape changes slowly; Philanthropy by its very nature is a rather conservative sector. But it does change, and it takes our effort and intention to help it change.  There are new search terms listed in Foundation Center online: Environmental justice. Climate change. Agrodiversity. Circus Arts. Those are subjects now listed by funders as areas of focus for funding, that weren’t there a few years ago. Let’s put “Art-science” on the list. – brain candy, eye candy

I’ve been enjoying the delightfully cerebral articles I find in the science journal, “Nautilus.” This sample article on the topic of Why Nature Prefers Hexagons is exemplary of this journal’s artful appeal to the senses and cultural relevance, and with firm roots in science.

Venus' Flower Basket, Dmitri Grigoriev, Shutterstock

Venus’ Flower Basket, porous skeleton of a sponge.  Photo: Dmitri Grigoriev, Shutterstock, the website of this sumptuous art-sci magazine, was originally funded by a grant from the John Templeton Foundation. The website offers its articles online for free (seven free articles per month), but for full access, also offers subscriptions for both hard copy and online versions. There is also a store for its delightful science-inspired visual art offerings.

To explore a bit of background on science journalism over the past few decades, check out this NYT article. Nautilus quarterly issues are organized around particular topics, such as “Boundaries,” “Adaptation,” “Attraction,” “Space,” “Stress,” “Identity,” “Scaling”…

Some sample article titles, for further enticement:

Why It’s Hard for Black Holes to Get Together

Why We Swim in Quarries

Describing People as Particles Isn’t Always a Bad Idea

Dive in, enjoy, and tell me what you think.