Carnegie Airborne Observatory visits our area


Carnegie Airborne Observatory image of rainforest trees; different colors represent different spectral fingerprints. Click picture to enlarge. Image: Carnegie Institution for Science.

The Carnegie Institution for Science is a unique private organization devoted to advanced study of the earth, life, and the universe. The pioneer cosmologist Edwin Hubble (“Hubble constant”), geologist Charles Richter (“Richter scale”), geneticist Barbara McClintock, and many Nobel laureates from several different disciplines are or were Carnegie investigators. The institution has instruments orbiting Mercury, is a lead partner in constructing the world’s biggest telescope in Chile, and has one of the world’s most sophisticated ecological monitoring devices, the Carnegie Aerial Observatory (CAO). This is a two-engine 20-passenger plane that Greg Asner and colleagues has fitted with millions of dollars worth of specially-designed lasers and spectrometers. It can sample hundreds of thousands of hectares of forest per day, using LIDAR to build a 3-dimensional model of the forest’s trees with 8 cm resolution. At the same time as it acquires LIDAR data, it also samples the spectral properties of light reflected from the vegetation, gathering reflectance information at hundreds of different wavelengths (colors). This spectral data gives information about the chemical and physical properties of the leaves, and also provides a spectral fingerprint that can later be matched to field-collected spectral fingerprints from known species of trees. Some  trees have such distinctive fingerprints that they can be identified to species with this data; more commonly, they can be identified to genus, though sometimes only to family. The detailed structural, chemical and taxonomic data acquired by the CAO would be impossible to gather at the landscape level by any other method, and Greg’s work is dramatically expanding the range of questions that ecologists can ask about forest ecosystems.


Carnegie aerial observatory rainforest image: 3-D Lidar combined with spectral signal. Image: Carnegie Institution for Science.

Last year Greg had planned to use our mosaic of forests as reference sites for a study of Andean forests on different geological substrates and elevations.  Greg and his partner Robin Martin visited our Rio Zunac Reserve, his flight plans got approved by the Ecuadorian authorities, and everything seemed ready to go, but in the end he was not allowed to bring the plane into the country. This year, however, Greg was able to bring the plane in for a more modest ten-day study of Amazonia. The plane’s home for those ten days was the military base in Shell, a town in the upper Rio Pastaza watershed near our Rio Anzu Reserve. One of the CAO’s flight transects covered a two-kilometer wide strip from west to east (high to low) through our area, perhaps including parts of up to four of our reserves. This will be a very valuable data set that will teach us a great deal about the structure and diversity of these forests. However, it will take about a year to fully process the data, so we’ll have to be patient.


The Carnegie Airborne Observatory parked at the Shell military base. Our reserves are in the mountains in the background. Click picture to enlarge. Photo: Matt Scott.

The president of the Carnegie Institution for Science, Matt Scott, is a well-known geneticist and serious photographer. He came t0 Ecuador last month to fly with Greg, but first he wanted to visit some of our reserves. Our endangered Black-and-chestnut Eagles (Spizaetus isidori) were nesting again in our Rio Zunac Reserve after last year’s tragic nest failure, so this was a once-in-a-lifetime opportunity to observe the species as it went about its business.

I picked him up in the Quito airport. The trip from Quito to Banos was picturesque as always. The glacier of Cotopaxi was covered in a layer of fresh volcanic ash, and small puffs of ash and vapor were still rising up from the crater as we drove past it.


Cotopaxi’s glaciers covered in fresh ash. Click picture to enlarge. Photo: Matt Scott.



Close to sunset as we neared Banos after passing through a rainstorm. Click picture to enlarge. Photo: Matt Scott.

The next day we had an appointment with the Black-and-chestnut Eagles at 10am-11am. Our guards told us the parents  usually brought prey to the baby at that time, but were otherwise rarely seen around the nest. The nest is about 3-4 hours away from the road, after a forty minute drive from Banos, so we had to get up early and rush out there. It was hard to keep up  a good pace, since beautiful things kept distracting us. Still, we managed to get to the nest observation spot at almost exactly 11:00, and sure enough, there was the adult in the nest, along with the chick and something dead. The adult flew off almost immediately but shortly returned to feed on the prey item while the sated chick slept. The other adult was also nearby and both called frequently. We spent an hour watching them. It was a wonderful thing to see.


This was the view when Matt got to my house to start our trip to the Rio Zunac. Volcan Tungurahua with a lenticular cloud against a crystal sky, a great way to start the day. Click picture to enlarge. Photo: Matt Scott.


Morning fog over the Rio Pastaza. Click picture to enlarge. Photo: Matt Scott.


Black-and-chestnut Eagle (Spizaetus isidori) at its nest in our Rio Zunac Reserve. Photo: Matt Scott.


We saw several Highland Motmots. Photo: Lou Jost/EcoMinga.


Torrent Ducks on the Rio Zunac distracted us throughout the day. Click picture to enlarge. Photo: Matt Scott.


We found this crazy katydid at the end of our walk. Click picture to enlarge. Photo: Lou Jost/EcoMinga.


Butterflies and hesperids taking salts from the sand along the Rio Zunac. Photo: Lou Jost/EcoMinga.


Matt chills out in the Rio Zunac after our hike. Click picture to enlarge. Photo: Lou Jost.


The next day we went to our Rio Anzu Reserve near the Shell airport and the CAO. That reserve is not very rich in big stuff, but there are so many interesting small things that it is hard to take ten steps without stopping for photos. We eventually got to the Rio Anzu river and the magnificent fossil-bearing limestone formations capped with ladyslipper orchids (Phragmipedium pearcei). Though it was getting late, Matt asked to stay longer. I always like to hear that from a visitor!!


Matt photographing the limestone. Click picture to enlarge. Photo: Lou Jost/EcoMinga.


The limestone formations along the Rio Anzu, covered with orchids. Click picture to enlarge. Photo: Matt Scott.


Phragmipedium pearcei, a ladyslipper orchid, on the limestone. Click picture to enlarge. Photo: Matt Scott.


Riodinid butterfly in the Rio Anzu Reserve. Click picture to enlarge. Photo: Matt Scott.


Large hairy caterpillar. Click picture to enlarge. Photo: Matt Scott.


Me in bamboo forest along the Rio Anzu. Click picture to enlarge. Photo: Matt Scott.

Then we went to the military base to see the CAO. Security was tight and the military were not eager to let a pair of muddy rubber-booted gringos walk through their installations. Nevertheless we were able to talk our way through the multiple layers of officials who scrutinized us. But we didn’t want to ruffle any feathers so when we finally got to the plane, we just took a quick look at it and went back (still under military escort, but actually a very friendly one).


CAO at the military base. Click picture to enlarge. Photo: Matt Scott.

By the time we got to Greg and Robin’s hotel in nearby Puyo it was already dark. Greg was sitting at a table outside working on maps in his laptop, and he showed me the transects he had flown so far. I went back to Banos that night but Matt stayed and got to fly in the CAO over the following days. Lucky man!


Matt (left) and Greg happy to be in the air. Click picture to enlarge. Photo: Matt Scott.


The Rio Pastaza broadens and meanders as it leaves our mountains and enters Amazonia. Click picture to enlarge. Photo: Matt Scott.



The Amazon basin from the CAO. Click picture to enlarge. Photo: Matt Scott.



More of the Amazon basin from the CAO. Click picture to enlarge. Photo: Matt Scott.

Matt, thanks very much for your visit! It was an honor for us to show you our forests.

Lou Jost

Fundacion EcoMinga



Earth Day: High school students from Aldo Leopold’s alma mater spend a week in our Cerro Candelaria forest

Aldo Leopold’s 1949 book, “A Sand County Almanac”, was one of the first voices of the environmental consciousness that began to awaken in response to the post-World War II rise of man’s destructive power. The founder of Earth Day, Wisconsin Senator Gaylord Nelson, was deeply influenced by his writings.

Aldo Leopold wanted humanity to develop a land ethic, one that respected plants and non-human animals. He wrote:

“When god-like Odysseus returned from the wars in Troy, he hanged all on one rope a dozen slave-girls of his household whom he suspected of misbehavior during his absence.”

“This hanging involved no question of propriety. The girls were property. The disposal of property was then, as now, a matter of expediency, not of right and wrong. Concepts of right and wrong were not lacking from Odysseus’ Greece: witness the fidelity of his wife through the long years before at last his black-prowed galleys clove the wine-dark seas for home. The ethical structure of that day covered wives, but had not yet been extended to human chattels. During the three thousand years which have since elapsed, ethical criteria have been extended to many fields of conduct, with corresponding shrinkages in those judged by expediency only.”

“…There is as yet no ethic dealing with man’s relation to land and to the animals and plants which grow upon it. Land, like Odysseus’ slave-girls, is still property. The land-relation is still strictly economic, entailing privileges but not obligations.”

Perhaps (just perhaps) our ethical sphere has been extended a bit since Leopold wrote those words, but we have a long way to go. We spend less and less time in nature, to the point where most people today do not even know what real nature is. Intact ecosystems are now so rare that the vast majority of people will never experience them, much less fall in love with them. This visceral love of nature is the only thing that can drive people to sacrifice their own comforts to protect it.

Dr John L. Clark, who holds the Aldo Leopold Distinguished Teaching Chair at The Lawrenceville School, Aldo Leopold’s alma mater in New Jersey, is as much in love with nature as anyone I know. He has started a program to bring his high school biology students (ranging in age from 15-18 years old) to our reserves in Ecuador, to try to ignite this passion for real nature in the next generation.

John is an old friend of mine who used to be a Peace Corps volunteer here in the 1980s. He is now a famous botanist specializing in gesneriads, the African Violet family. He has published several monographs on gesneriad genera and has discovered many new species. Two years ago, as a professor at the University of Alabama, he brought a college biology class to our Rio Zunac Reserve to set up two quarter-hectare plots, in which every tree bigger than 10 cm in diameter was sampled, tagged, and identified. Dr David Neill from the Universidad Estatal Amazonica helped set up that plot and identified the trees. In the process they found what turned out to be two new species of Magnolia trees, and John discovered a new gesneriad in the genus Columnea.

Now in his new position at The Lawrenceville School, he has done the same thing with a dozen of his high school students, joining with David Neill again to set up a quarter-hectare plot in our Cerro Candelaria Reserve last month. It was a daring project, very unusual for an American high school.

Some of his students wrote about their experience. Here is Kaimansa Sowah’s essay, which she titled “Botanizing!”:

“Never had I seriously considered ecology or botany or even entomology as a field of interest until our trip to Cerro Candelaria on the eastern slopes of the Andes in Ecuador. Arriving in Quito on a Saturday morning with many missionary groups crowding the lines at immigration, I questioned if our work in Ecuador would have any real impact on the community. How could plant identification transcend traditional community service? It would not be until I was sitting around a fire at our high camp sipping tea made from recently collected crushed foliage of a Lauraceae we had found earlier, barely communicating sufficiently in my middle school Spanish that I managed to realize the profound importance of our trip to Ecuador.”

“The hike up to camp was brutal to say the least. Many of us had never hiked before and mounted on our backs were 50-pound packs with silica gel for preparing museum specimens, M&Ms (which would be our lunch for several days), and personal belongings. Our frequent stops for “Botanizing!” only heightened the difficulty level. Our expedition leader Dr. John Clark lights up at a fallen Gesneriaceae leaf, so throughout the hike and the trip as a whole, he was never short of excitement as our paths were lined with rare and new species. Fortunately, the view of mountains perfectly scattered, parting only for the rapids leading to and from waterfalls, fuelled our strenuous walk to the camp. The view never ceased to amaze us, and many of us still fail to believe its reality.”

“It was not until we began work on the plots that each of our own individual love for botany and plant life was established. Divided into groups of three, we established and inventoried tree diversity in a 0.25-hectare permanent plot. With the help of Tito, our guide, friend, and resident tree climber, we identified trees based on vegetative features (e.g., leaf patterns, leaf arrangement, smell), recorded DBH (diameter breast height), tree height, and tagged each tree with an aluminum label. Our field journals appeared something like this: “tree 4, subplot 5, 25 meter height, 18 cm DBH, simple-alternate leaves with milky sap (Moraceae?).” On the first day we found a cherry tree (Prunus sp.) that had never been observed by our resident scientist and tree expert, Dr. David Neill who is a professor of biology at the Universidad Estatal Amazónica. Many of the trees were challenging to identify, which only further affirmed how much biodiversity surrounded us. During a lunch break, we played a plant identification game where we were divided into teams and given Al Gentry’s book “A Field Guide to the Families and Genera of Woody Plants of North west South America.” Each team was timed in their ability to identify foliage to family. All of us being extremely competitive, we quickly held our leaves to the light using our hand lenses, crushing and smelling, and rapidly blurting out names like “Piperacae!” Euphoribacae!” “Melostomatacae!””

“Along with our own Dr. Clark were resident entomologists and ecologists who shared their love of biology. We also met the director and founder of the EcoMinga foundation, Lou Jost who is a theoretical mathematician, ecologist, and botanist who specializes in the study of orchids. We were surrounded by vast amounts of unique talent, which greatly sparked our own interests. Besides the fieldwork, we were able to connect and talk with our guides. They soon became our friends, and it was through conversations with them that we realized how grateful they were for our interest in visiting their reserve. No, as a sixteen-year-old girl, I had never thought of biodiversity research as one of my interests. And I cannot say whether it was our guide giving us hints during the scavenger hunt with his ability to identify plant families from meters away, or the sheer look of ecstasy when “Ranger”, also known as Dr. Clark, and Dr. Neill sat around their pressed leaves dumbfounded at a new species, or Darwin [Recalde]’s ability to navigate the maze-like mountains and carting us up steep hills. Nonetheless, this trip has piqued my interest and I suspect that botany and biodiversity will play a large role in my future.”

Eloise White wrote of her experience:

“…When I first signed up to travel to Ecuador with the School, I expected a week of light hiking, bonding with new friends, and great food, all coupled with the occasional botanical reference. While the food was indeed fantastic, the intensity of the trip took us all by surprise on the first day in the field, when we embarked on a challenging four-hour hike to our camp. It was not until after we finished showering in the beautiful waterfall and sat down at dinner to prepare our field notebooks for our work in the tree plots the next morning that I realized the importance of the work that we would accomplish during our time in the forest.”

“When we reached the plots bright and early the next day, we received instructions, and my group quickly fell into a rhythm of tagging trees with bright orange tape and communicating with our local guides who were climbing to the canopy of the trees, a task that gradually became easier as our Spanish improved. Each time that our guide, usually some 30 feet high in a tree, would cry “Ten cuidado!” the three students in my group would jump back and wait for an unidentified specimen to come crashing to the ground. That first day, in the moments that I spent with Dr. Clark, tagging and pressing plant samples into pages of newspaper, his excitement surrounding new and rare species was absolutely contagious. I found myself eager to memorize the names of plant species, to identify which types of bark had latex, and to distinguish simple leaves from compound leaves. Even now, I find myself so grateful to Dr. Clark and the other scientists accompanying us in the forest because they showed me what it means to be passionate about a specific field of study, something that I hope to do as I move forward in my Lawrenceville career, the college process, and my life.”

“…My Spanish teachers at The Lawrenceville School have always stressed the importance of experiencing the language abroad in order to truly further my understanding… Between trying to ask our guides to scale a certain tree to obtain a specimen and sitting around our campfire late in the night, telling ghost stories and jokes with Jordi and Darwin, I was constantly speaking Spanish. The pure exposure to the language coupled with the locals’ willingness to help me practice provided me with a unique opportunity to further an area of interest which I had not previously devoted much attention to. Furthermore, partially overcoming the language barrier opened the group up to an irreplaceable chance to form lasting friendships with locals, a memory that I will forever treasure. Lawrenceville constantly stresses the importance of expanding our horizons, and I can attest that in communicating with and working alongside unfamiliar faces, the twelve of us expanded our own world views significantly.”

“Before embarking on our journey, our teachers made it clear that our accommodations would be far from luxurious. We were told us that we would be perpetually damp, sweaty, and dirty, all of which later proved true. However, I will be the first to say that the view from our wooden cabin base camp without windows, doors, or even walls was extraordinary, rivaling that from any mountain getaway or island. When we summited Cerro Candelaria (3800+ m), while it was extremely challenging and put both our bodies and minds to the test, the breathtaking outlook from the top instantly made our hard work worth it.”

“Overall, my work and experiences in Ecuador were once-in-a-lifetime opportunities. They opened my eyes up to an entirely new scope of interests, people, and awareness. For example, as I previously planned on dropping out of Spanish for my senior year, I have changed my mind and will continue to advance my understanding of the language, hopefully into college. As I begin the college search, I have been relentlessly pestering my counselor about which schools have the best programs to study abroad while working with the science department. I attribute these shifts in my interests to my recent experience in Ecuador.”

Vivienne Gao expresses the very real physical challenges of this trip:

“Honestly, if I had known our expedition to Ecuador involved so much hiking, I probably would not have signed up. I’ve always been more comfortable in the water; I prefer swimming over running and am generally more athletic when I am not on land, so the minute I found out that our first hike to low camp would take roughly four hours, I definitely had my doubts…The day was hot, but not unpleasant, but I still kept my hair in braids to keep it off my neck. Once we began our hike however, the physical exertion made the heat borderline unbearable. We all carried large Osprey backpacks with our personal belongings, and these bags were not only heavy but also didn’t breathe well. Sweat happily gathered between my back and my pack, soaking through my shirt so that when I finally peeled the pack off, my shirt still clung to me as a dog’s fur clings to it, dripping, rinsed after a soapy bath in the backyard. The hike was mostly uphill, but the terrain varied. We were slopping through mud, climbing over rocks, and wading through streams, sometimes on level ground and sometimes on downhill slopes, but everything led us upwards eventually.”

“I remember seeing the cabin for the first time after three or so hours of hiking and thinking that this was the best moment of my life. I had fallen behind with a couple friends, so the rest of the group was already in the cabin waiting for us. As I slowly trudged up the hill, humoring the impressive cramp in my right calf that had formed over the duration of the hike, I congratulated myself for completing the hike, a feat that I considered the most difficult thing I’ve ever done. Little did I know that in the days to come, I would experience hikes many times more difficult than this one, a prospect far beyond my wildest imaginations…”

“Twenty minutes before we reached low camp, my small group of hiking companions and I had come across waterfall, the same waterfall that would host our daily shower and laundry trips. After reaching camp, everyone, me included, was excited to wash the salt and dirt off their bodies. The idea of showering in a waterfall enticed me, but my legs caved at the thought of hiking another twenty minutes to the waterfall, yet I went anyways. The waterfall became my favorite place and I went everyday after that.”

“I came back from Ecuador having learned more about my physical and mental limits, surprised at how hard I could actually push myself. I lost eleven pounds but earned so much more in experience and memories. The trip is something I will never forget, and who knows, maybe someday I’ll return, ready to face the challenges I faced this time and conquer them.”

As his students noticed, John Clark was at least as excited as they were:

“I am often asked how I know when something I come across is a new species. It is important to note that describing a new species is a process that is collections-based, requires several formal criteria outlined by the International Code of Nomenclature (ICN), and is contingent on a peer-reviewed publication. It is considered by some biologists (e.g., L.E. Skog who co-chaired my PhD committee) as “bad botanical etiquette” to say something is new without data. Nevertheless, outlined here are four species that I am confident have not been previously described. My doctoral dissertation resulted in a monographic revision of Glossoloma (Clark 2005). This is a group of plants that I dedicated more than a decade studying and when finished, I expected that there would be an occasional new species that would represent something that was not included in the monograph (Clark 2005). For example, Karyn Cichocki observed a new species of Glossoloma in 2007 when assisting me on an expedition in Ecuador. An additional new species was described with a student as a result of an expedition in Colombia (Rodas & Clark 2014). What I did not expect to find in Cerro Candelaria was a new species of Glossoloma every 500 meters in elevation change. I found three new species of Glossoloma between our base camp and the high camp. We also discovered a an undescribed species of Drymonia, which is a group that Laura Clavijo and I have studied together for more than eight years. I directed Laura’s dissertation committee (2007 to 2015) and together we have published more than eight papers on Drymonia. Thus, the four undescribed species featured in Figure 1 [below] are based on ongoing studies of museum specimens, extensive fieldwork, and comprehensive review of taxonomic literature. The remarkable discovery of biodiversity featured in Figure 1 is an example of the urgency and need for additional studies in the Neotropics.”

“There are also rare species from Cerro Candelaria that I did not expect to find. Two collections represent populations that were not previously known. The rarest plant that we found was Columnea bivalvis (photo below, D and E), which was previously only known from a single population (Amaya-Márquez & Clark 2011). [Note added by LJ: That original population was found in what is now our Rio Machay Reserve.] Drymonia ignea (photo below, A and B) is endemic to the eastern slopes of the Andes and was previously only known from 5 populations (Clark 2013). Never have I seen more than a few individuals of Drymonia ignea growing together and along the ridgeline there were multiple areas of ten or more individuals.”

The Lawrenceville School students not only gave us their friendship and enthusiasm but also brought the gift of electricity to our research stations. I’ll save that story for a separate post.

The Lawrenceville School staff who visited us: Baptiste Bataille, Jennifer Mayr (her husband is related to famous evolutionary biologist Ernest Mayr!) and John L. Clark.

The Lawrenceville School staff who visited us: Baptiste Bataille, Jennifer Mayr (her husband is related to famous evolutionary biologist Ernest Mayr!) and John L. Clark.

The essay excerpts used here are from John’s fuller version of this story which will soon be published by the magazine “Gesneriads”. They are used here with John’s, the school’s, and the magazine editor’s permission. Thanks John, and thanks Lawrenceville School students, for a wonderful cultural exchange and exciting scientific discoveries! Your enthusiasm and that of your students inspires us and makes our work feel worth the trouble. Lawrenceville School students, you literally walk in the footsteps of the great Aldo Leopold, and I hope that like him, some of you can help the earth face the challenges that your own generation will witness.

EcoMinga also thanks the World Land Trust and their donors Puro Coffee, Naturetrek, and PricewaterhouseCoopers for funding the Cerro Candelaria Reserve, and their donor Noel McWilliam for the funds to build the research station where these students, and many other students and scientists, stayed.

The World Land Trust’s “Forests in the Sky” appeal continues to expand the protection of this area.

Lou Jost

Estimating the diversity of an ecosystem based on an incomplete sample

The genus Lepanthes is one of the most diverse plant genera in our area, with more than 100 species just in the upper Rio Pastaza watershed. We keep discovering more new species, even after twenty years of intense surveying. This is typical of tropical floras and faunas. It is a real challenge to try to estimate the diversity of a rich group of organisms like this. We need the help of clever statistical methods! Painting: Lou Jost.

The genus Lepanthes is one of the most diverse plant genera in our area, with more than 100 species just in the upper Rio Pastaza watershed. We keep discovering more new species here in our reserves, even after twenty years of intense surveying. This is typical of tropical floras and faunas. It is a real challenge to try to estimate the diversity of a rich group of organisms like this. We need the help of clever statistical methods! Painting: Lou Jost.

Ecologists and conservation biologists often need to quantify the diversity of some community of animals and plants. In the tropics, where there are many rare species, this can be hard to do, because there are so many rare species; a random sample from the population will miss many species. My friends Phil DeVries, Tom Walla, and Harold Greeney were still finding additional species even after five years spent systematically sampling the butterfly diversity of a single research site in the Amazon basin of Ecuador. This is typical of tropical habitats. So biologists need the help of mathematicians and statisticians to help estimate the true diversity based on incomplete samples.

The person who has done the most to advance this aspect of statistical theory is Anne Chao, who I am lucky to have her as a friend and coauthor. She has figured out how to make use of information in the sample to tell us how incomplete it is, even when we don’t know how many species there are in the actual population. I have written about this subject before, when she and I figured out how to compare the diversities of two or more ecosystems based on incomplete samples from each of them (with a version in Spanish here). This is a common task, but biologists had always been doing it wrong. That post was about estimating the number of species in each community. However, there are many reasons why that number is not the best measure of biological diversity or complexity. Not only is it the hardest number to estimate accurately (because there are always unseen species), it also ignores the relative abundances of species, and those abundances have an important effect on the ecosystems’s complexity. An ecosystem with one very abundant species of birds and nine very rare species is less complex than an otherwise-similar ecosystem with ten equally common species of birds. An individual bird in the first ecosystem can be quite sure that the next bird it meets will be a member of the very abundant species. A bird in the second ecosystem will be completely uncertain about which of the ten species it will encounter next.

The amount of uncertainty in the species identity of the next individual encountered is actually something that can be quantified exactly if we know the relative abundances of each species in the ecosystem. In 1948 Claude Shannon, the inventor of information theory, showed that this uncertainty is equal to the entropy function that had been derived by the physicists Boltzmann and Gibbs in the late 1800s. Shannon’s discovery led to the wide use of entropy in other disciplines, including ecology. Shannon’s entropy measure became the most commonly used abundance-sensitive measure of biodiversity. The entropy is a simple function of the relative abundances of each of the species present in the population. Ten years ago I showed that this entropy function needs to be transformed by taking its exponential before it can be interpreted as diversity (Jost 2006, 2007, 2010). But whether we are interested in entropy or its exponential, in biology we still have to estimate these quantities from incomplete samples, so we don’t know the true relative abundances of the species in the population, and we don’t even know exactly how many species there are in the population.

Anne had been thinking about this problem for more than thirty years, and recently came up with a beautifully elegant solution for estimating entropy from small samples. The solution makes clever use of the information contained in the sample about the unseen species in the population. I wrote briefly about it here. This month she was asked by the journal Methods in Ecology and Evolution to write a blog post about her solution. You can read her full blog post here, complete with essential links to more information. Here are some excerpts, with my notes in brackets:

Estimating Entropy from Sampling Data

In practice, the true number of species and their relative abundances are almost always unknown, so the true value of Shannon entropy must be estimated from sampling data. The estimation of this seemingly simple function is surprisingly difficult, especially when there are undetected species in the sample. It’s been proven that an unbiased estimator for Shannon entropy doesn’t exist for samples of fixed sizes.

The observed entropy of a sample or the ‘plug in’ estimator, which uses a sample fraction [the abundance of the species in the sample, divided by the size of the sample] in place of the [true] relative abundance of species [in the population], underestimates the entropy’s true value. The magnitude of this negative bias can be substantial.

For incomplete samples, the main source of the bias comes from the undetected species, which are ignored in the plug-in estimator. An enormous number of methods/approaches have been proposed in various disciplines to obtain a reliable entropy estimator with less bias than that of the plug-in estimator. The diversity of the approaches reflects the wide range of applications and the importance of bias-reduction.

My Introduction to Alan Turing’s Statistical Work

Around 1975 (when I was a graduate in the Department of Statistics, University of Wisconsin-Madison) my advisor at the time, Bernard Harris, suggested that an “attractive and absorbing” (his original description) PhD thesis topic would be to develop an ‘optimal’ entropy estimator based on sampling data. He thought Alan Turing’s statistical work might prove to be useful and hoped that I could tackle this estimation problem.

Alan Turing memorial statue in Sackville Park, Manchester, UK ©Lmno

Alan Turing memorial statue in Sackville Park, Manchester, UK ©Lmno

However, at that time I didn’t even know who Alan Turing was! Although I started to read two background papers by I. J. Good (links below) about Turing’s statistical work, I couldn’t fully digest the material in the short time available. So, I didn’t work on the entropy estimation problem for my PhD thesis; instead, I derived some lower bounds for a variety of diversity measures. Ever since then, however, entropy estimation has fascinated me and has been in my mind/thoughts, and I regarded it as my ‘unfinished thesis’ topic.

The Building Blocks of My Entropy Estimators

According to I. J. Good (Turing’s statistical assistant during World War II), Turing never published his wartime statistical work, but permitted Good to publish it after the war. The two influential papers by Good (1953) and Good and Toulmin (1956) presented Turing’s wartime statistical work related to his famous cryptanalysis to crack German ciphers. After graduation, I read these two papers many times and searched for more literature. It took me a long time to fully understand these two papers especially Turing’s statistical approach to estimating the true frequencies of rare code elements (including still-undetected code elements), based on frequencies in intercepted ‘samples’ of code.

“Mathematics, rightly viewed, possesses not only truth, but supreme beauty, a beauty cold and austere, like that of sculpture.” -Bertrand Russell ©Lmno

“Mathematics, rightly viewed, possesses not only truth, but supreme beauty, a beauty cold and austere, like that of sculpture.” -Bertrand Russell ©Lmno

The frequency formula is now referred to as the Good-Turing frequency formula. Turing and Good discovered a surprisingly simple and remarkably effective formula that is contrary to most people’s intuition. The formula proved to be very useful in my development of entropy estimators.

One important idea derived from the Good-Turing frequency formula is the concept of ‘sample coverage’. Sample coverage is an objective measure of the degree of completeness of the intercepted ‘samples’ of code elements. The ‘sample coverage’ of a sample quantifies the proportion of the total individuals in the assemblage that belong to sampled species. Therefore, the ‘coverage deficit’ (the complement to sample coverage) is the probability of discovering new species, i.e. the probability that a new, previously-unsampled species would be found if the sample were enlarged by one individual. Good and Turing showed that for a given sample, the sample coverage and its deficit can be accurately estimated from the sample data itself. Their estimator of coverage deficit is simply the proportion of singletons (in this case species with only one individual) in the observed sample. This concept and its estimator play essential roles in inferring entropy.

A Novel Entropy Estimator

A species accumulation curve (SAC) shows the cumulative number of species as a function of sample size. In the figure below we see the expected curve when individuals are sequentially selected from a community with a given number of species, with relative abundances.

The first breakthrough in my search for an estimator of Shannon entropy was the realization that entropy can be expressed as a simple function of the successive slopes of the SAC.
[Anne is probably the only person in the world who would have noticed this!] The curve’s successive slopes show the rates at which new species are detected in the sampling process. I had found a novel way to estimate entropy via discovery rates of new species in a SAC and these rates or slopes are exactly Turing’s coverage deficits for varying sample sizes!

The statistical problem was then to estimate the expected slopes or coverage deficits for any hypothetical sample size. Good and Turing’s approach provided the coverage deficit estimator for the expected slope of the sample that has been taken. All of the expected slopes for smaller sample sizes can be estimated without bias from statistical inference theory. However, there is no unbiased estimator for the expected slopes for sample sizes greater than the sample taken. These slopes are usually dominated by rare undetected species whose effect on entropy cannot be ignored. So, the burden of entropy estimation is shifted onto the estimation of the expected slopes for sizes greater than our sample.

The second break-through step to solve this problem was also attributed to the wisdom of Turing and Good, who showed that the number of singletons carry much information about the number of undetected rare species. I slightly modified their idea to use both singletons and doubletons to estimate the hard-to-estimate slopes by my modified Good-Turing frequency formula.

With the collaboration of Lou Jost and the simulation/programming work of Y.T. Wang, we published in 2013 the novel entropy estimator based on the derived slope estimators [open access to full text]. Our extensive simulations from theoretical models and real surveys generally showed that the new estimator outperformed all the existing estimators. It took me over 35 years to derive the optimal estimator for my ‘unfinished thesis’, so I have been calling it my entropy ‘pearl’. (The novel entropy estimator along with other related estimators can be calculated online.)

Doing Research is like Carving Jade

…As the old saying goes: “doing research is like carving jade, we are never satisfied with what we have until it is perfect”. This is also my advice to anyone starting their career in academia. The topic of entropy estimation has attracted and absorbed me for more than 35 years, and hopefully the novel estimator did yield an ‘optimal’ solution, if it’s still not perfect.

In her blog post Anne also explains how this method generalizes to the estimation of a wider class of diversity measures based on generalized entropy, a problem she and I had been working on for ten years. See her original blog post for details.

Anne’s contributions to the mathematics of biology are one of the reasons why Sebastian Vieira and I recently named a new orchid after her. Thanks, Anne, for your very fruitful collaborations over the years!

Lou Jost


Chao, A. and Jost, L. (2011). Diversity measures. Sourcebook in Theoretical Ecology (eds. A. Hastings and L. Gross). Berkeley: University of California Press.

Chao, A. and Jost, L. (2012) Coverage-based rarefaction and extrapolation: standardizing samples by completeness rather than size. Ecology 93: 2533-2547.

Chao, A. and Jost, L. (2015) Estimating diversity and entropy profiles via discovery rates of new species. Methods in Ecology and Evolution 6: 873–882.

Chao A, Wang YT, Jost L (2013) Entropy and the species accumulation curve: a novel entropy estimator via discovery rates of new species. Methods in Ecology and Evolution 4: 1091-1100.

Jost, L. (2006) Entropy and diversity. Oikos 113: 363–375.

Jost, L. (2007) Partitioning diversity into independent alpha and beta components. Ecology 88: 2427–2439.

Jost, L. (2009) Mismeasuring biological diversity: Response to Hoffmann and Hoffmann (2008). Ecological Economics 68: 925–928.

Two more new frogs discovered in our Rio Zunac Reserve

Pristimantis sacharuna. Photo: Mario Yanez.

Pristimantis sacharuna. Photo: Mario Yanez.

Our Rio Zunac Reserve has been an endless source of new discoveries of plants and animals. I’ve written in the past about the discovery of new magnolia species, new melastomes and orchids, and new frogs. Last month herpetologists Juan Pablo Reyes (who is also our reserve manager), Carolina Reyes, Maria Perez L., and Mario Yanez, (who is also an EcoMinga director and head of the National Institute for Biodiversity), have published two more new frogs from this reserve. One of the new species, Pristimantis pinchaque, was discovered at 1600m elevation in the immediate vicinity of the scientific station that we built there some years ago with the help of the IUCN-Netherlands and the Netherlands Postcode Lottery, while the other new species, Pristimantis sacharuna, was discovered farther up the trail from the station, at 2200m. That makes four new species discovered so far in this reserve, joining Pristimantis ardyae and Osornophryne simpsonii.

Pristimantis pinchaque. Photo: Mario Yanez.

Pristimantis pinchaque. Photo: Mario Yanez.

Pristimantis pinchaque is apparently very rare; it has not been seen since the first two specimens were found in 2008, even though many herpetologists have visited the site since then. It is named after the Mountain Tapir, Tapirus pinchaque, a charismatic and endangered mammal which lives in the same forest. Pristimantis sacharuna is also apparently very rare, with only two specimens found in four years of investigation. It is named after the “duendi” or mythic forest man of indigenous legends.

The story of these discoveries was covered nicely by the national press. The country’s largest newspaper, El Comercio, even made an interactive article that lets readers see the diagnostic traits of each frog by clicking on different parts of its anatomy, and in a special feature for their “Planet” section, they also published a nice diagram to help readers distinguish the frogs:

And there is still more to come! We have two more new frog species being described right now, from survey work supported by a donation from Henri Botter and Ardy Van Ooij of the Netherlands.

These investigations are collaborative efforts between EcoMinga and the National Institute of Biodiversity, the Zoological Museum of the Pontificia Universidad Católica del Ecuador, and the Fundacion Oscar Efren Reyes. We’re excited to have such distinguished collaborators, and we are eager to see what surprises still await us in these very special forests.

Lou Jost

An eagle specialist team visits our Black-and-chestnut Eagle (Spizaetus isidori) nest

The star of the show, the latest picture of our baby Black-and-chestnut Eagle. It continues to grow very fast. Photo: Angel de Pazo Carballo.

Here is the latest picture of our baby Black-and-chestnut Eagle (Spizaetus isidori). It continues to grow very fast. Photo: Angel de Pazo Carballo.

Juvenile Black-and-chestnut Eagle in our Rio Zunac Reserve, several years ago. Photo: Luis Recalde/EcoMinga.

A different juvenile Black-and-chestnut Eagle (Spizaetus isidori) in our Rio Zunac Reserve, several years ago. Photo: Luis Recalde/EcoMinga.

Very little is known about the endangered Black-and-chestnut Eagle (Spizaetus isidori), so Fausto Recalde’s discovery of its nest in our Rio Zunac Reserve was exciting news for ornithologists. Ecuador’s Harpy Eagle project team came to visit our nest a few weeks ago, with Fausto Recalde as their guide. The ease of access to this nest makes it an especially good research opportunity, and the team asked our guards to make at least weekly observations and photos of the nest and baby. The team was led by Ruth Muñiz López, a Spanish biologist living in Ecuador. She is a famous expert on Ecuador’s giant eagles, especially the Harpy (Harpia harpyja).

Top row, left to right: Luis Astudillo, Fausto Recalde. Bottom row, left to right: José María Gil Sánchez, Ruth Muñiz López with baby Breogan, y Ángel de Pazo Carballo.

Top row, left to right: Luis Astudillo, Fausto Recalde. Bottom row, left to right: Ángel de Pazo Carballo, Ruth Muñiz López with baby Breogan, and José María Gil Sánchez.

Ruth, her baby Breogan, and Fausto Recalde on the trail to the eagle nest. Photo: Angel de Pazo Carballo.

Ruth, her baby Breogan, and Fausto Recalde on the trail to the eagle nest. Photo: Angel de Pazo Carballo.

Afterwards Ruth gave our staff and interested Banos residents an interesting talk on the Harpy and Black-and-chestnut Eagles.

Ruth giving a talk to us about eagles. Photo: Lou Jost/EcoMinga.

Ruth giving a talk to us about eagles. Photo: Lou Jost/EcoMinga.

Although the magnificent Harpy gets most of the global attention, our beautiful Black-and-chestnut Eagle is actually far more endangered globally, because it lives only in a narrow band along slopes of the northern Andes. The Harpy, in contrast, is sparsely but widely distributed throughout the lowland rain forests of Latin America, from Mexico (where it is close to extinction) to northern Argentina. The Black-and-chestnut Eagle is easier to see than the Harpy, though, because it soars often, while the Harpy spends most of its time below the canopy trying to ambush something.

Captive Harpy Eagle. Photo: Wikipedia.

Captive Harpy Eagle. Photo: Wikipedia.

Ruth told us that these big eagles can live for many years. There was a famous Harpy in a zoo in Tuxtla, Chiapas, Mexico, which had been captured as an adult and lived to be at least 48 years old. I visited that bird back in the 1980s, and it looked ancient even then. No one has any information about how old a Black-and-Chestnut Eagle can live. Perhaps we can help fill in that information by careful observations.

Oldest known Harpy Eagle. Tuxtla zoo, Chiapas, Mexico, 1985. Photo: Lou Jost.

Oldest known Harpy Eagle. Tuxtla zoo, Chiapas, Mexico, 1985. This bird had been shot as an adult by a hunter. Its broken right wing is visible in my photo. It was given to naturalist and zoo director Miguel Alvarez del Toro, and stayed alive in this zoo for many years. It finally died in 1990. I’ve put more of my photos of this bird at the bottom of this post, since there do not appear to be any other photos of it on the internet. Photo: Lou Jost.

The Harpy team spends a lot of time rescuing and rehabilitating captive or injured Harpies, and Ruth told us about some of those experiences during her talk. For example, someone captured a baby Harpy Eagle by cutting down its nest tree. Ruth and her team heard about it and went to visit. They somehow convinced the person to return the eagle to the wild. They then built an artificial nest in the jungle near the original fallen nest tree, and the person who had captured the eagle placed it in the new nest. Even though three months had past, the adult returned, bringing food to the baby! The excited baby fluttered around so much that it fell out of the nest. But eventually everything went well and the baby fledged.

Captive Harpy Eagle. Photo: Lou Jost/EcoMinga.

Captive Harpy Eagle. Photo: Lou Jost/EcoMinga.

Our guards were impressed by Ruth’s falconry work, keeping Quito’s new airport free of collision-prone birds. They had seen her on TV using tame raptors (birds that, for one reason or another, were too tame to release into the wild) to chase off the other birds, and they imagined it was from some far-away place.

Ruth at the Quito airport keeping the skies safe for airplanes. Photo: L. Astudillo.

Ruth at the Quito airport keeping the skies safe for airplanes. Photo: L. Astudillo.

Ruth is trying to get a satellite transmitter to track our young bird after it leaves the nest. She and her team have experience with this, and they assure me that there is no danger to the young bird. They’ve also connected us with a photographer who would like to document the nest. Another photographer, Mark Wilson, is writing a book on the large eagles of the world and will also be coming to photograph it this month. Knowledge is a key to conservation, and we hope to cautiously take advantage of this opportunity to learn as much as we can about our most spectacular and most endangered bird.

Lou Jost

Earlier posts on the Black-and-chestnut (or Andean) Eagle, Spizaetus isidori:

Monkey killer: Black-and-chestnut Eagle (Spizaetus isidori)

Black-and-chestnut Eagle nest discovered here at last!!!!!!!!!!

Better views of our Black-and-chestnut Eagle nest

Black-and-chestnut Eagle (Spizaetus isidori) again, in a different reserve

Our baby Black-and-chestnut Eagle is growing up fast!

Rio Zunac update: predator and prey

More photos of the oldest known Harpy Eagle:

Oldest known Harpy Eagle. Tuxtla zoo, Chiapas, Mexico, 1985. Photo: Lou Jost.

Oldest known Harpy Eagle. Tuxtla zoo, Chiapas, Mexico, 1985. Photo: Lou Jost.

Oldest known Harpy Eagle. Tuxtla zoo, Chiapas, Mexico, 1985. Photo: Lou Jost.

Oldest known Harpy Eagle. Tuxtla zoo, Chiapas, Mexico, 1985. Photo: Lou Jost.

Oldest known Harpy Eagle. Tuxtla zoo, Chiapas, Mexico, 1985. Photo: Lou Jost.

Oldest known Harpy Eagle. Tuxtla zoo, Chiapas, Mexico, 1985. Photo: Lou Jost.