In a paper published in The American Naturalist in 1992, Jessica Gurevitch, Laura Morrow, Alison Wallace and Joseph Walsh presented the results of what was, arguably, the first meta-analysis in ecology, of field competition experiments published in six leading ecology journals over a 10-year period. Gurevitch and colleagues found that, overall, competition had a strong effect on biomass, but there was a lot of heterogeneity across different organisms. Twenty-four years after the paper was published, I spoke to Jessica Gurevitch about her motivation to do this study, her memories of doing the work that went into this paper, and the growth of meta-analyses in ecology and evolution.
Citation: Gurevitch, J., Morrow, L. L., Wallace, A., & Walsh, J. S. (1992). A meta-analysis of competition in field experiments. The American Naturalist, 140(4), 539-572.
Date of interview: 2 December 2016 (via Skype)
Hari Sridhar: I wanted to start by asking you about your motivation to do this piece of work. I looked up your publication profile and I came to know that in your PhD, which was about 10 years earlier, you had done work on competition in plants and then you continued to work on plant ecology for the next 10 years. How did the idea for this synthesis paper come about?
Jessica Gurevitch: Well, that’s sort of several questions at once. So, you’re correct, I did a big field experiment on plant competition and on the question of, “Was competition important in limiting local distribution of a particular plant species?” And, of course, I was immersed in the competition literature as a PhD student. You had to read everything, and be aware of all the work that had been published on competition. And back at that time,what I found was that – this is going back a long time ago – most of the work on plant competition had been done in greenhouses, i.e. all controlled environment experiments, and not so much had been done on field experiments. That was one of the things that made my dissertation work unusual. And also there were several other novel things about that work; quantifying competition experimentally along an environmental gradient was also new. But starting around the same time that I was doing my PhD work, a lot of people started doing field experiments on competition in a wide range of systems. And that was very interesting. Not only on plants, but on many other species. I’m a plant ecologist. I’ve always focused my interest on plants. I’ve expanded from there, but the heart of my work and the heart of my interest has always been in the ecology of plants. Also, at that time, there was very long standing and very… I don’t know the word… a very passionate argument going on about whether competition was really important in nature? And there were alternatives to competition; predation could be so intense that competition never had much effect in nature. So there was this general question, “Is competition something that is occurring as an important factor in structuring natural communities?, and what evidence do we have that competition occurs of a substantial magnitude and is effective in structuring communities and affecting populations?, and so on, in nature”. So, that was part of the motivation for doing that paper. But then the other part is the meta-analysis part. Should I talk about that part?
HS: Yes.
JG: Okay. So the scientific question that came up at that time was, “Is competition important in nature?”, with the evidence coming from the tremendous amount of work that was being done since I had first initiated my own experiments, up until 1989, when I first got the idea of doing that meta-analysis paper. What evidence could we get from all of these papers and all of these different systems for how important competition was in nature? That was an open question. And as a plant ecologist, it seemed obvious to me that competition was important, but you still need evidence. At the same time, in 1989, I was struggling. I had not gotten a federal research grant. I was an Assistant Professor and I’d published some things, but maybe not enough. And I was struggling and concerned about tenure. So, I applied for a fellowship, in part to stop the tenure clock. By having a fellowship and taking a leave of absence, I postponed the tenure clock for what amounted to a year. I was gone for a semester. I had a fellowship at the Arnold Arboretum of Harvard University to work on leaf shapes, which was another interest of mine. I was using their large collection of trees that had been collected from all over the world and were growing in the Arboretum in Boston. And so I was living in Boston and I had the freedom to just think about things broadly. And I read an article in The Boston Globe about a study that had been done in the educational literature that asked the question: “Were boys really smarter than girls in mathematics?” which seemed inherently very interesting to me. And the answer was, “No, they’re not.” It was based on a study of children in elementary school grades. And what they had done was combined the results of many different studies using a meta-analysis. Well, I had never heard of meta-analysis before, and I was instantly electrified by the concept. And they explained in this newspaper article that this was a relatively new statistical technique to combine the results of separate studies to reach general conclusions and to resolve apparent discrepancies and results among studies. And I thought, “Wow, this is something!” I immediately was knocked off my seat. I thought this is something that we could use to resolve these questions about the effects of competition in ecology, because there had been many studies with many different results. And I thought this would be an amazing tool to introduce to ecology. I had never heard of it. It had not been used in ecology up till that point. So, I immediately dropped everything I was doing, and sadly neglected the leaf shape study, and went off to learn everything I could about meta-analysis. My husband (Todd Postol; we were newlyweds at the time), was immensely encouraging and told me I absolutely should pursue this idea. So I literally dropped everything, ran to the library and started finding everything I could. I started reading everything I could. Of course, nothing was online in those days. I took out every book I could – those of interest were mostly in the social sciences at that time. And one of the books I read was the book by Larry Hedges and Ingram Olkin on the statistics of meta-analysis. And I just pored through that book and thought, “Okay, this is something we could do in ecology.” I learned everything I could about the statistics and practice of meta-analysis from the books I could find. I was very concerned with the statistical assumptions: what made it possible to legitimately carry out this kind of analysis? This book by Hedges and Olkin (Statistical Methods for Meta-Analysis) was clearly an authoritative source on how to do this right. I went back to my study on leaf shapes, but continued to think about this idea for a meta-analysis in ecology. So that was the initial motivation, and I can go on with the rest of the story.
HS: In your CV, I read that, in 1991-92, you got a grant to do this work. Could you tell us a little about that?
JG: Yes. When I went back to Stony Brook after this leave of absence, I was very fired up to do this meta-analysis. And I realized it would be a really large project, so I asked a number of students I knew whether they’d be interested in collaborating with me on it. And I found a few students who were. This was a couple of grad students (Laura Morrow and Alison Wallace), and an undergraduate student who was in my giant (>500 students) Intro Bio class and always sat in the front row. Anyway, we met every week, photocopied huge numbers of papers to include, and were making a lot of progress on this meta-analysis, and figured out how to extract data from graphs by digitizing them (Morrow’s idea and Walsh’s steady hand in doing it) and how to carry out the analyses, guided by the Hedges and Olkin book. We submitted the paper to The American Naturalist, and after a lot of explaining and revision, it was accepted. I began to give talks explaining what we had done, and met up with the usual objections to meta-analysis that people have experienced in other fields as well (combining apples and oranges, etc.).
Around the time just before it was published, while we were still working on the final version, I realized that there were complications and issues for ecological data that were not being addressed in the social science literature that I was looking at. And in fact, they weren’t addressed in the medical meta-analysis literature either. So I wanted to develop new methodology, new statistical methodology, for addressing some of these more advanced issues. At that time, I was on a National Science Foundation panel – I don’t even remember what the panel was about, but some ecology panel. I met with a program officer (Jim Reichman) on one of the breaks and told him about the idea for expanding this work, to be able to develop the statistical methodology or meta-analysis for ecological data. And he was very supportive and very interested in this. And he said, “Well, why don’t you write a small grant, not a full size regular research grant, but a small grant for something that’s of immediate relevance that you want to kind of get going?” And so that’s what I did. I wrote the proposal, and I got that funding, and that was the start of more methodological development.
HS: I’d like to hear more about your initial foray into doing meta-analysis with your students and what the challenges were then.
JG: There were many challenges, but it was a very exciting and fun time. I’m trying to remember the status of all of the people. One of them was a PhD student (Alison Wallace), one was a student who was going to get a master’s degree – that was Laura Morrow – and then there was this undergraduate, Joe Walsh, who had been a Wall Street stockbroker and decided that was totally boring and didn’t want to lead his life being a stockbroker. So he came back to school. He was in my Intro Bio class because he had never had bio – he wanted to become an evolutionary biologist – and he was back in university studying for a biology degree. He would sit in the front row, and just look intensely interested all the time. So I asked him, “How would you like to be involved with a research project?” So the group of us met weekly, we divvied up projects and we started figuring out how we go about doing this. How do we gather the data, how do we extract the data, how do we ask the questions in a quantitative way? This took a lot of collaborative work and a lot of thinking and creativity on everyone’s part. Everyone was very critical for the success of that project. And we worked on it for a long time. It took us about a year; we had a lot of frustration, a lot of humour. Some of the humour was when we could not figure out what people had done in their studies. You know, they didn’t report their data in a way we could use it and we said, “What we need to do is we need to commission the services of a statistical psychic to understand what was in these papers! We just had a lot of fun with it, we worked very hard, and we finally put this big paper together. One of the ideas that I had was: if meta-analysis was going to be introduced to ecology, it should be introduced in an important way – take a very important topic that a lot of people would be interested in, and something that would get a lot of attention, because otherwise people would ignore that this was something that was an important new tool that had broad applications. So that’s why we tackled such a big project. It was really very, very large and challenging to do that analysis. And of course, in those days, nothing was electronic. We were photocopying all the articles from, you know, from the print versions of the journals that we got at the university library and you know, doing things in a way that you wouldn’t do them now. And the statistics also is much more simplistic than what we would do nowadays.
HS: By “group”, do you mean the authors?
JG: Yes.
HS: You mentioned earlier that you read the Hedges textbook, and I notice that you’ve later written papers with him. At the time when you were trying these things out for the first time, did you consult people in, say, social sciences or medicine, to get a sense of how to these analyses were done?
JG: No, I figured it out myself with this group of students. But when I started feeling that we needed further methodological development, I contacted Larry Hedges, not to help with the meta-analysis that I had done, which was largely completed at that time, but to get him interested in working with me to develop new methods. Of course, there was no email then either,so I called him on the phone and reached him and said, “I’d like to, you know, ask questions and potentially collaborate”. And he said, “I don’t want to do any more meta-analyses. I’ve just spent too much time doing meta-analyses.” And I said, “No, I don’t want to ask you to do any meta-analyses. I want to develop new statistical methods.” And then he was really interested and we ended up collaborating on a number of different projects and becoming very good friends.
HS: Before I ask you more about this paper, I just want to step back a bit and ask you about your interest in plant ecology. Could you tell us where that came from? Did you have this interest for a long time before you started research?
JG: Yes. I grew up in Brooklyn in New York City, and I lived in an apartment house during my childhood. And when I was in high school, my family bought a little row house that had a tiny little backyard. And I thought, well, I would like to grow some plants back there. So I started trying to find out about plants. I had a very urban background, and I didn’t know anything about plants, and I didn’t know any people who did anything with plants. So I would walk around my neighbourhood and there were these little old Italian grandfathers who had amazing gardens in their front yards. They had these tiny handkerchief front yards, they grew fig trees and grapes and all kinds of interesting things. And I would just stop and look and talk with them and ask them what they were doing. And so I learned something about growing plants from the advice and then I read everything I could, and I started growing things and I loved it. I just loved working with plants. So this was when I was about maybe 14 years old, 14 and 15 years old. And then when I was in High School, I remember thinking: what kind of university am I going to apply to and what kind of job might I have? I certainly did not know anyone who was a scientist. Science seemed like something for boys, not for girls. And I certainly didn’t know anyone who did science, male or female. And I didn’t know anyone who did things that were not, you know, in an urban environment. And I was really, you know, unsure how to develop this interest. So, I thought, well, maybe plant breeding. I knew people bred plants and they bred plants for horticulture and food. And I thought, well, that, you know, maybe that’s … I knew I wasn’t going to be a farmer; that was not going to work. I thought, okay, maybe plant breeding would be a good thing. And so I took a class in Advanced Placement Biology, and the advanced placement courses, which now are ubiquitous, were just being introduced at that time. And so I took this AP Biology, in which we had a textbook by the famous paleontologist George Gaylord Simpson called “Life” and we went through the earthworm digestive system and all this totally boring stuff – apologies to the earthworms digestive biologists – but finally we got to biogeography and ecology and evolutionary biology and I thought, “Okay, this is it. I want to be a plant ecologist; I want to be able to look at plants and nature and understand them and understand what’s going on. And I want to be able to travel.” I was really fascinated with the possibility of travelling to different parts of the world to study and learn about plants. And so I just thought, “Okay, where can I learn about ecology?” This was before the first Earth Day. Nobody that I knew had heard the word ecology; it was a very unusual word. And when I was a senior in high school and looking for places that taught ecology, there were not many. And people kept thinking I was saying I want to study psychology, because the word sounded sort of similar. They’d never heard the word ecology. That’s how unfamiliar it was. So I mailed off for the course catalogs of all of the universities in the region and looked to see which ones had a rich ecology program and I saw that Cornell had a big ecology program. And so I applied to Cornell and was admitted—which caused a big stir in my huge and rather anonymous public high school. Robert Whittaker was there and other famous ecologists were there. And so that is how my interest developed in plants and ecology, but it came from the initial interest in just growing plants and spending time with plants and I decided I loved it then and I love plant ecology still to this very day. I love growing plants and thinking about plants. And I have been incredibly, incredibly fortunate to have travelled all over the world, as I dreamed as a young teenager, to do science and think about plants and see, you know, see places and think about ecology in different places. It’s amazing. It’s just been an amazing, amazingly fortunate thing for me.
HS: Coming back to the paper: You mention that, nine years earlier, there were two other syntheses of competition experiments – by Connell and Schoener. Did you decide to only use experiments from 1980 to 1989 in your meta-analysis, because of these earlier papers?
JG: No, it was a practical issue. Remember, at that time we were going and photocopying papers out of journals. And the idea in meta-analysis at that time was more to come up with a defined, unbiased list of papers that you would want to include and then search those out. We also only looked at a certain number of journals; major ecological journals. You wouldn’t do a meta-analysis that way now because there’s electronic access, and not only access to the journal articles, but to the databases for searching for articles. Those databases didn’t really exist, you know. Web of Science didn’t exist. Science Citation Index was just, it was in paper and it was very awkward to use; you couldn’t search for keywords. So it was strictly a practical issue and it was still at the very edge of our capacity to accumulate all of that information. So, strictly a practical issue.
HS: You chose six journals to search for competition papers. Can you tell us why these journals?
JG: Just what we thought would be the ones that would be most likely to have published papers that were field experiments on competition.
HS: Were all of these journals available in your university’s library?
JG: Yes, that’s right. And we just photocopied enormous numbers of articles and had to go through them very carefully to see did they really have the information that would be possible to incorporate in that meta-analysis.
HS: Did you take the formulas to calculate effect sizes and other metrics from the Hedges book? Did you modify them in some way?
JG: We largely followed the approach of the Hedges & Olkin book for sure.
HS: Do you remember how long it took you to put together the data, do the analysis and write it up?
JG: Well, I started reading about this, I first got this idea in April of 1989, when I was in Boston. And then I came back to Stony Brook in September, at the end of the summer and then started gathering this group of people and talking about the idea. So we didn’t really start the work on the project until the end of 1989. And then it took more than a year to gather all these papers together, figure out all of the tremendous technical problems and issues, and start to do the analysis. We did the analysis on spreadsheets, which again, you would never do now. All of that was happening in mostly 1990 and 1991. And then I wrote up the paper and submitted it, I don’t remember exactly when, but it was somewhere through 1991.
HS: How long did the writing take?
JG: The writing took some time.The main time that was involved, though, was in gathering the papers, evaluating the information, figuring out how to do this with ecological data,and doing the analysis.The writing definitely took some time, because we had to deal with this very large literature on competition and explain not only what we were trying to address in terms of the scientific question about competition and the field, but also what was meta-analysis and what were we doing with this data.
HS: Do you remember how you drew the figures for this paper?
JG: They were all drawn by hand. In those days, there was no other way to do it. So, I think what we did was, if I remember correctly, I think we got a scientific illustrator to draw them on Mylar sheets, which is what you did back in that time. I think that’s how we did it. Yeah, if I remember that’s what we did…but boy, I’m not really sure. Maybe that was, you know, maybe that was the beginning of being able to draw these things on computers, but I don’t remember exactly. It was right at the transition between these things being drawn with pen and ink and things being drawn on computers. So, I don’t remember how the figures were all done.
HS: If you don’t mind, I want to go over the names in the acknowledgments to get a sense of who these people were and how they helped.
JG: Sure. I haven’t looked at this in a long time. Let me see if I remember.
I’m looking at what I published in 92. I was also working on other projects at the same time. So that also ended up taking more time and I was teaching this giant Intro Bio class with 550 students. All right. I have the paper. Let me see what the acknowledgments were
HS: The first name is D Slice.
JG: Yeah, so that’s Dennis Slice (now very sadly deceased). Dennis Slice was our IT person. He had gotten a PhD in evolutionary biology, working with Jim Rohlf in my department and then was working as the IT person, running the computer systems and giving a lot of computational and statistical advice in the department at that time (he eventually went on to become a noted researcher and Professor in South Carolina and then at Florida State University). So one of the really challenging questions which Laura Morrow came up with a solution for was: we realized that a lot of the data was not published in tables; It was published in figures. And so we thought, how can we get this data? It’s in figures, what are we going to do? And she said, she had been taking a course with Professor Jim Rohlf, and he had mentioned that people could digitize figures and extract the data. And she said, we could do that with this paper, and then Dennis Slice helped us to actually figure out how to do that. And then Joe Walsh did a lot of that because he had amazingly steady hand-eye coordination. He did a lot of digitizing. So this was very much a collaborative effort.
HS: FJ Rohlf
JG: That’s Jim Rohlf, who is now a retired professor in my department; Professor Emeritus. And he had been doing a lot of work with digitizing for morphometrics, which he became very, very famous for. He was already famous for doing statistical work. So he had all of this digitizing equipment that he generously allowed us to use to digitize the figures.
HS: And then you thank a few people for providing additional data. This included E Bauder, A Desrochers, R Karban, W Morris and R Ryti
JG: So these are people we wrote to. We wrote to a lot of people where we couldn’t find the data. Many people did not answer us or they didn’t have the data anymore, but those people all provided us with additional data, so we could use the information from the experimental work they did.
And then there are people you thank for advice, comments and encouragement. The first name there is FS Chapin
HS: That’s Terry Chapin, who is a famous Arctic ecologist and very famous ecologist overall. So, what I started to do, as the paper started taking shape and we started getting results, I went on a kind of a promotion tour for meta-analysis and was giving talks in as many places as I could, to get people to start thinking about using this approach and what you could do with the approach and why this was important advance for ecological research and ecological science, to be able to synthesize the results of different experiments. A lot of different people gave me feedback and responded, some of them anonymous and some of them people I knew. Terry Chapin was one of those people who I met at that time who was very supportive and gave me good advice. Let’s see who else. Rick Karban is another one who I met at that time and he also shared data and gave good advice. The other people are people I already knew or people I met in the course of this. Dan Dykhuizen was a colleague, he is also now Professor Emeritus. He was a professor in my department, an evolutionary biologist who studies short-term evolution in bacteria and yet he was really interested, really supportive, really encouraging. So that was very, very important and helpful, you know, asking questions and giving good advice. Deborah Goldberg: she and I went to grad school together. And she’s always been interested in competition and in competition in plants. She’s a plant ecologist. And she had published a paper at almost the same time, a narrative review paper on competition, competition among plants and field experiments. And so we talked about that. Larry Hedges: Around the time that this was taking shape, I contacted him, met him, talked to him about our data, but the work that I did with him was on more advanced techniques that we did after this paper. Janet Morrison was a PhD student with me and she didn’t work on this project, but she and I worked on another meta- analysis subsequently. And she also gave feedback and helped with, looking at the writing and, was generally interested in the project. Todd Postol is my husband. When I first came up with this idea back in 1989, we had just gotten married, he was still working on his PhD, which is in History, and we were living in Boston, in Jamaica Plain, which is near the Arnold Arboretum – lovely neighbourhood. And, of course, I immediately told him about this idea and he was the most supportive. He said, “Drop everything. This is a career changer. Do it. This is terrific.” And he was just super encouraging. And, you know, we had a lot of criticism along the way and some really negative and conflicting things, and he was just like, “Don’t listen to that. This is just a great idea.This is going to be something that really changes your career.” And he was totally right, and very, very supportive. And we are still married so many years later, and coming up on our 32nd anniversary! [in November 2020] And now we have a son who recently graduated college and a daughter who is in high school, who just got her college admission acceptance [update in November 2020: she’s now a senior in college and graduating from Colorado State University in May 2021]. James Thomson was another faculty member who has since left Stony Brook but was also very interested and supportive.
HS: Would you say that this paper had a relatively smooth ride through peer review and was Am. Nat.the first place you submitted to?
JG: Yes, Am. Nat was the first place we submitted it to. That just seemed like exactly the right place for it. I don’t think it sailed through. I don’t think anything I’ve ever written has sailed through review. One of the issues that we had to address was this other paper that was coming out at almost the same time, that came out just a few months before this paper, by Deborah Goldberg. And we had to say: What was different about this paper? Why was it an advance? And, you know, what was non-overlapping? And also what was different between this paper and the Connell and Schoener papers? Connell and Schoener were, and are, very famous ecologists, and, you know, so they were very high profile people.What was new or different or worthwhile about this approach? What did we have to offer that was different from what had already been done? Why was it legitimate? So we had to make a strong case for that. But it wasn’t terrible.
HS: Given that this was completely new to ecology, were there a lot of comments in the reviews on the approach itself?
JG: A bit of it, but more of the emphasis was on competition and on the science in the review process. In the talks I was giving, a lot of the comments and objections and arguments were about the methodology – about the meta-analysis – but in the review process, I remember the main comments were about the issues regarding competition.
HS: You introduced meta-analysis into ecology. Do you know if there were other people in the field who might have been having the same idea around this time?
JG: Well, there actually was another paper published the year before, of a meta-analysis in ecology. It was a paper by an ornithologist named Jarvinen. It was about clutch size in a particular bird species. It was much simpler and smaller and a more narrow question. So that was really the first one in ecology. That was published in 1991. Another paper using meta-analysis in ecology was also published in 1992, when ours was and a highly influential review paper on the use of meta-analysis in ecology was published in 1995. I think these techniques are so powerful and so important and profound that other people would have come across them and introduced them probably not that long afterward, had I not done it. But what we did was sort of establish the framework for how you might use these techniques in ecology. So I think we were influential in that sense. But whether other people were really starting to work on these at the same time, or, you know, maybe shortly afterwards: certainly, in the following years, meta-analysis in ecology started being taken up exponentially.
HS: When you spoke about how you came across meta-analysis in the newspaper article, you said that the article mentioned that it was a new tool.Was it also new to Social Science and Medicine, at that time?
JG: It wasn’t that new in those fields at that time; it was new to the general public. And it was certainly new to me. It was first introduced in the Social Sciences in the 1970s, the late 1970s, and almost immediately afterwards in Medicine. I did not know about that medical literature for a while afterwards. So, I was really following the methodology more in the social sciences, but they really started using it in the mid to late 1970s. If you’re a specialist in those areas, and were reading the cutting edge research, you would have known about it. But they were not well known more broadly. So, you know, by 1989, that’s not so long for this approach to start reaching a more general audience and to reach beyond the fields in which it was first initiated and developed, to start influencing other disciplines.
HS: At the time when the paper came out, can you tell us a little about how it was received? Did attract a lot of attention?
JG: That’s really interesting. As I mentioned, I was then an Assistant Professor and I was very eager to establish myself, not only for personal recognition, which, of course, is good and appealing, but also because I was so excited about this approach, that I wanted people to recognize that this was something that lots of people could find very useful. And incidentally, right at around that same time, I found out that I had just gotten this big grant for doing field research through NSF. So everything was really wonderful and I ended up getting tenure – of course, wonderful in retrospect; at the time, it was not so certain. Anyhow, back at Stony Brook I would go over to the medical school library, which was not on the main campus, and look up in the giant books of the Science Citation Index, which was printed in a multi-volume set on very, very thin, like, almost transparent paper, because it was all printed out—“online” did not exist. And I would look up every couple of months to see – did anyone cite this paper? And at first, the only citations for the first couple of years, the only citations to the paper were only referring to competition. And they said, “Okay, Connell and Schoener did studies on combining research on competition and Deborah Goldberg did a review on competition and this paper also – Gurevitch et al. – did a paper on competition.” And I was like, “Oh, come on, come on you people, you have to recognize we introduced this important new approach”. It took quite a while. It took several years and I think a lot of going around and giving talks and saying, “This is something you really need to pay attention to. This is like a whole new approach to thinking about generality in science”; and a lot of controversy and a lot of argument about its legitimacy and the methodology and so on. There was a real lag until many people started picking up on the fact that this was a new technique that would really change the way we thought about synthesizing results across experiments, to reach a more general understanding.
HS: It’s been cited over 600 times. Do you have a sense of what it’s mostly gotten cited for over the years?
JG: I think a lot of it is for meta-analysis. Now that issue of how important is competition in nature is completely resolved. Nobody is saying, “Well, maybe competition doesn’t really occur in nature.” I think we really were able to address and resolve the question with this broad synthesis across so many different taxa and systems and so on. But now often, a lot of the citations are to the methodology of doing meta-analysis, although, curiously, we wouldn’t do a meta-analysis using those old methods, which we used at that time. We wouldn’t do that anymore. We would use more advanced methods.
HS: You’ve already said a little bit about this, but could you say a little more about what kind of an impact did this paper have on your career?
JG: Oh, it totally changed my career. Yeah. My husband Todd was right. So, this will go back a little bit further. When I was an undergraduate, I had no statistics at all. I didn’t take statistics. That was an oversight on the part of me and of my advisors, I guess. I also didn’t study evolution! Like, why would they have not told me to take evolution? What did I know! I wanted to learn about plants. Anyway, as a graduate student, I took a couple of courses on statistics that were rather cookbook and, you know, not all that interesting at the time. And then I was doing this large field experiment, which we talked about earlier, and I needed to know more statistics in order to analyze the data. And I realized that the obvious statistics were inadequate. So I started talking to a statistician who was an Assistant Professor in the university I was studying to get my PhD – University of Arizona – and – maybe being too wordy about this – the issue in that case was I had been measuring the plants in my experiment over a period of almost two years, with repeated measurements over time. And when I wanted to analyze the data, the advice was, “Well, just forget all those measurements and look at the end point.” I said, “I’ll be damned if I forget all those measurements which I killed myself to get in this field experiment”, which was so challenging, physically as well as intellectually, out in this remote range land. It was really hard to get all those measurements. And I said, “There must be something I can do with it”. And this person who I was consulting with, whose name was Ted Chester, said, “Well, there’s this method called repeated measures analysis that people don’t use much”, and I was like, “Okay, tell me about it.” So we figured it out,we wrote a paper together on repeated measures analysis and I got really hooked on statistics. I began to realize that figuring out how to use statistics to answer ecological problems is not a cookbook thing; this is really challenging. Ecological data is different. It’s more complicated in really interesting ways. And it just opened up a whole intellectual world for me of the intersection between statistics and science. That interest remained with me, and when I got to this idea about meta-analysis, I was really interested in how we can use this method in ecology, what makes sense scientifically, what gives you legitimate answers statistically, what are the limitations…all of those kind of substantive questions became really compelling and intriguing to me. And that led to this collaboration with Larry Hedges, who is a really amazing statistician. His interests are in the Social Sciences, but he’s a brilliant statistician. And so that opened up the doors to interacting with people across other disciplines. I started collaborating with and interacting with and learning from people in many different fields. And eventually that led me to a wider and wider range of interactions and collaborations. It just opened up a major broader kind of scientific knowledge and approach that totally changed the kind of scientist I was.
HS: In terms of your career, did this paper have any sort of direct impact? Did it help build your reputation in the field?
JG: Well, the tenure thing was a real issue. So there was at least one person in my department, who I will not name, who was very strongly against meta-analysis. In my department, we typically give a research talk as part of the process of coming up for tenure, and I talked about this project, this particular meta-analysis, because I thought this is something that’s really of broader importance than just the specific kinds of things I was working on. This had wide ranging importance. That’s what I felt and that was true. But this one person in particular was very strongly opposed to the idea of meta-analysis – and still is, actually – and made the tenure case challenging. So what I did was I went and talked to Professor Sokal (now deceased), and he was a National Academy member and very high profile and everyone respected him tremendously. And I talked to him about it, and he said this sounded like a perfectly legitimate approach. I was doing the work in a rigorous manner and he supported it and that saved my neck. He was so influential. So the tenure thing was one thing. There was undoubtedly a great deal of gender bias in the difficulties I faced in being awarded tenure and promoted—my application was rejected twice and only on the third time was I successful—this has never happened to men in the department, and they did not all have the most iron-clad cases. In terms of my reputation: absolutely. It changed my work from being, you know, a more narrowly focused ecologist to being someone who was getting recognition across a much broader array of different areas in science and, beyond the specific ecological systems I was researching. People were recognizing and using this work in many different areas. So that greatly increased my recognition. Another thing that greatly increased my reputation was writing a textbook. Writing a textbook makes a big difference because a lot of people who are not reading your particular research papers get to know who you are. So those two things put me in a very different category of scientific reputation, which is no doubt why you’re interviewing me right now, because you’ve heard of me.
HS: The textbook was published the year after the paper. Was the motivation to write this textbook linked to doing this meta-analysis on competition?
JG: This is the plant ecology textbook?
HS: No, I was referring to The Design and Analysis of Ecological Experiments.
JG: Ah, that’s different. No. So, I also wrote a plant ecology textbook and that made a big difference. The Design and Analysis book – I had the opportunity, also connected with the Arnold Arboretum, to make contact with scientists in Japan. And I went to Japan to continue the work I had been doing at the Arnold Arboretum, but also to attend the International Congress of Ecology at Yokohama. And Sam Scheiner who I had known from my postdoc days – he was getting his Ph.D. at the same time I was a postdoc at University of Chicago- also went to this INTECOL meeting. And we spent some time travelling and just talking together during that meeting, and we both felt that there were a lot of advanced techniques in statistics that would be very useful to ecologists. And he was also very interested in statistical applications and ecology. And we thought, you know, we’re not all going to be getting Ph.D.s in statistics, but we still need to be able to access these more advanced techniques. And so from that meeting, at that meeting, we thought, well, we could put together a book. We can’t do this ourselves but we can put together a book where we invite people to contribute to it and do an edited book with these advanced techniques and make it really accessible and, yet, do it in a way that wouldn’t just get people to do it off the shelf, but to really understand how to use these more advanced techniques appropriately. One of the reasons I think that book was successful, besides the fact that it had a lot of useful methods and we made it accessible, was that we work very hard to edit the book, and to go back and forth with the authors so that the book had a consistent feel and was written at a consistent level and with a consistent structure. So it had a coherence that a lot of edited books don’t have. A lot of edited books are a random assortment of papers that are very different in most ways. So this came across as more of a textbook type contribution. So that’s how that book got going.
HS: Today,would you say that, that the main conclusions of the of this paper still hold true with regard to what you say about competition?
JG: That’s a great question.The main, the big, message, that competition is widespread and important in nature, and that it has large effects across many different systems and many different organisms, that certainly still holds. The specific things that we found out: It would be really interesting actually to update this, not only to update the methodology but to update the synthesis with the vast amount of work that’s been done subsequently. That would be really interesting to find out.That’s really an empirical question that you’re asking and until the work is done, you don’t know what the answer would be. But I think the fundamental finding that competition is very widespread, very important and of substantial magnitude, in nature, that, I think, would not be changed.
HS: After this paper, have there been any other meta-analyses of field experiments of competition?
JG: Not that I know of, but I couldn’t say that for sure.
HS: Have you gone back to doing this again with, you know, more data and with the new methods?
JG: We have not repeated the basic meta-analysis that we did then.One of the things that was the most similar to that was I did a meta-analysis with Larry Hedges and Janet Morrison looking at the interaction between competition and predation in field experiments. We did that subsequently. That was to introduce new methodology but also to see does the presence of predators or herbivores alter the outcome of competition in field experiments. And we found out that, yes, it does.
HS: If you were to redo this today, what would you do differently?
JG: Well, we would do a number of things differently. So first of all, we would search much more broadly. We would not restrict the studies to a certain number of journals, we would just, you know, use scientific databases to search whatever had been published in different journals. So we would have a much wider range of studies, but then it would be too many studies to handle, so we would have to have to narrow it down in some way I guess. We used fixed effects models in that paper, which were the only statistical models that were available at that time. Now we would you use mixed effects, we would do much more complex modelling, and we would use meta-regression and not just heterogeneity tests. So we would use different statistical approaches as well as different search and inclusion approaches. We would also record and report the data differently, given the recent emphasis on reporting standards and Open Science.
HS: In terms of the emphasis itself: you had a list of questions that you asked in this paper, with regard to competition. You think those still remain among the most important and interesting questions to ask today?
JG: Some of them are still controversial, and they’re still around. Some of them, I think, are well resolved. So, for example, the question – Is competition more intense in high than in low productivity habitats? There have been subsequent papers on that,and there’s still quite a bit of argument about that issue.
So, yeah, there are still some things that are unresolved. Some things we will never have the same kind of data. We had questions about the effects of competition and carnivores. And most of the data on carnivores are on things like odonates. Yeah, you can do experiments on competition in dragonflies, but you can’t do much in the way of experiments on competition among tigers. You’re never going to be able to get that kind of data. So some of those things are still unresolved, some of them will always be unresolved and some of them could probably bear more research to update what the current state of knowledge is.
HS: You mention a few unpublished manuscripts in this paper. I just want to ask you if these were published subsequently. You talk about an unpublished manuscript by Gurevitch and Morrow, where you talk about bias in people conducting and reporting on competition experiments and you say that will be examined elsewhere. Was this been published later?
JG: No, no, we never did anything with that. Unfortunately, we never completed that.
HS: In this paper you use biomass responses to competition, and then you say that other measures of response such as responses in density and mortality are examined elsewhere.
JG: We looked at a few of those in a couple of book chapters and what not. I think some of that data was looked at in that book that we were talking about – the Design and Analysis book – we looked at some of that data. But frankly, it became overwhelming. This was just such a big effort, it became really overwhelming to do more with it. And as we were talking about, I was also involved with a lot of other things going on at the same time. So some of that we never really looked at again, we never published it, which is kind of a shame because we went to a lot of work to collect that data. But what we went with, was the data where we had the most information. So we had the most information about biomass.The other things were smaller and more constrained data sets and we never published that. But in Laura Morrow’s Master’s degree, in her thesis, she did include it, I guess.
HS: You also talk about non-independence of data. There’s a section in the paper where you discuss different sources of non-independence and say that meta-analysis has spurred the exploration of the consequences of the non-independence of effects, but the matter is far from resolved. Is this something you focused on later when researching the statistics of meta-analysis?
JG: I have not done that much with it myself, but other people have. Non-independence is an issue across ecology,both in primary studies as well as in meta-analysis. And it’s important in meta-analysis way beyond ecology. So, there are a lot of different kinds of non-independence in data. That is a really interesting problem, it is not fully resolved, but people have done a lot of work on it. For example, phylogenetic non-independence is something that now people have published not only papers on but also software to address phylogenetic non- independence.And other issues of non-independence have been the subject of further developments in meta-analysis. But I haven’t done that. I haven’t done that kind of work much myself. But it’s one of the examples where when you were doing meta-analysis, you have to think about issues that if you are doing a narrative review or an informal review or, you know, some other kind of approach to making sense of the literature, you don’t have to think about it, you don’t address it, those are all things that are hidden under the carpet, so to speak. And many issues, when you are doing a meta-analysis, require a lot of thought and investigation, and that’s one of them. And I think it’s really valuable to be able to have to think about those things.
HS: I just want to go back to that, you know, that unpublished manuscript on bias. I remember looking up your software MetaWin, where you do talk about how to deal with bias. So when you say you didn’t work on this further, was it in relation to this particular data set?
JG: No, I didn’t work on that. Well, let me see. So, where was that?
HS: P. 544
JG: Let me go back and see what exactly I was referring to. I don’t remember.Yeah, okay. Oh, yeah. So that’s the question of publication bias, and publication bias has been addressed a lot in the meta-analysis literature. And, yeah, I’ve talked about publication bias. It’s only one of the kinds of bias that may be relevant to research synthesis. But it’s one that gets a lot of attention. And I haven’t done a lot with that myself, but other people have been very concerned with how you detect the magnitude of publication bias. Something I have been involved with though, that comes out of that, is the whole issue of open science. So I’ve had some very recent publications, with collaborators, on issues regarding open science, and reporting both significant and non-significant results is one part of that. So one of the ways to resolve the issue of publication bias – i.e. non-significant results don’t get published and significant results are the only ones who are published, so therefore any summary of the literature is going to be biased because it ignores the unpublished non-significant results – is to publish non-significant results! And this big push for open science that I have been involved with, that is one of the motivations for instituting standards for open science. It’s not the only motivation, but one of the benefits of open science is to emphasize the importance of publishing data more completely, including both significant and non-significant outcomes.
HS: Another thing you mentioned about the methods is that you used a fixed effects model, because a mixed model wasn’t developed at that point. You also mentioned that later on you have used a mixed model. Was that something you developed yourself?
JG: No. The mixed models and random effects models were being developed in this by the statisticians who were developing methodology in the meta-analysis literature. And I used techniques that other people had developed. Mixed models was something that I did work on with Larry Hedges quite a bit. So that was something I made some contribution to, but now, people are much more interested in and using meta-regression, which incorporates both fixed effects and random effects in a more comprehensive statistical model.
HS: You also say that the statistics that you use assumes normal distribution of the primary data, but you don’t know if the original authors actually tested the data to meet this assumption. And you don’t know how robust the statistics are to violations of these assumptions. Today, are there ways to deal with situations where you don’t know whether the primary data is normally distributed?
JG: I think you still often don’t know much about the primary data. But I think those issues are more likely to be a problem when you have small sample sizes. I think the issues were less with the primary data and more with the distribution of the statistics you were using and the effect sizes you were using in meta-analysis. And that’s why you use things like log response ratio and standardized mean difference, because the assumption is,if the sample sizes are large enough, then you’re going to not be too far off of the assumption of normality. But one of the issues for lots of data, not just in ecology, but in other fields as well,is sample sizes are often very small. And that’s a real question. There are a lot of fields in which the sample sizes are really small, including a lot of the molecular and genomics data, where you often have two or three replicates. In ecology, they’re a little bit better than that. They’re still too small, but the limitations of statistical power assumption of normality and many other issues when you have very small sample sizes are not fully recognized in a lot of disciplines, and ecology is far from the worst. I think in the molecular and genomics data, people are much less aware of those issues.
HS: Towards the end of your paper, you say “the study makes it clear that more organisms need to be studied in underrepresented systems. We know quite a lot about competition among terrestrial plants and among many molluscs, and almost nothing about terrestrial herbivores are carnivores of any kind. Intraspecific competition needs to be examined much more often. And we need to devise ways of studying mobile animals, particularly in aquatic systems without enclosing them. And the relationship between how much organisms compete in nature and how much they compete in field experiments remains unknown for these and other reasons discussed about.” For these particular gaps that you identify,do you have a sense of whether we have more information today?
JG: I think we have somewhat more information. So we mentioned the issue of publication bias. A lot of people talk about publication bias; that’s not my term But I have come up with a term that I call research bias where certain things are very,very well studied and other things are not so well studied. And I think research bias is still a very major issue. So I did a paper a few years ago with a postdoc Ed Lowry, who was the lead author on that paper, we did it with a group of students in my lab group, looking at data on biological invasions. And we found that the data on biological invasions has very, very strong research bias. For example, there was a dearth of published papers on biological invasions in the tropics. That was not a meta-analysis. It was a systematic review to look at the scope of the literature that was addressing questions about causes and impacts of biological invasions. So I think research bias is still a very big issue. And it’s not only because people are poorly informed or something like that, it’s because a lot of systems are inaccessible and difficult to study,they are in places and with organisms that either people don’t have the expertise to study, or there are no ecologist to study them in those places. Research, in general in ecology, is biased toward a small number of geographic regions – North America, Western Europe, Australia, New Zealand, Hawaii – that accounts for an awful lot of ecological research on anything – biological invasions or competition or anything. And many other areas have many fewer people who are available to study those things and many fewer people who have expertise in those organisms, and they may be just really difficult to study. So studying competition among bats – wow, really difficult! They’re nocturnal, they fly,they are just really tough. Plants: much easier to do – they sit there and let you manipulate them. So, yeah, there are still big issues with research bias. On the other hand, ecologists are definitely studying more different kinds of organisms and more different settings than any other people in any other scientific research field. So, if you look at other scientific research fields, they have a much narrower focus in terms of the number of species they study and so on. So we’re doing pretty well in ecology, but we still have a lot of gaps.
HS: You also highlight poor reporting in papers, and emphasize that basic statistical data, standard deviations and sample sizes, need to be mentioned clearly, and that graphs need to be simple. Would you say that the quality of reporting has improved today?
JG: It has improved somewhat. So this is something I’ve been ranting about for almost 30 years now. And there are still big problems. I’m still working with people and pushing and I’ve published publish various things since then and continuing up to the present to try to improve the quality of reporting of information, including the very recent efforts on open science, to get people to report data in ways that are more transparent and less biased and more accessible, so that we can carry out unbiased and complete research syntheses, but also so that we can evaluate primary papers in the most logical way. So, these are still issues, there are still problems. It’s better, it’s improved, there’s more attention to these issues, but it has been a very long struggle to improve reporting standards.
HS: The final point you make in the conclusions is about the complexity of experiments.You say it’s often a trade off between complexity and sample sizes for each of the sub-treatments. You suggest that one way to deal with this is to do simple experiments,and use meta-analysis to be able to compare different treatments and different effects. What are your thoughts on this today?
JG: It’s still a real issue. It will always be a real issue that there is a big trade off in ecological experiments between statistical power and being able to address all the questions you want to address in a particular experiment. And one of the challenges is just being able to get published. So if you do a really simple experiment, it may be more difficult to get that primary study published. But on the other hand, when you do complex experiments, then your sample sizes are necessarily smaller because you’re going to have more treatments and more different kind of things you did. So, those are all real challenges. I should also mention with respect to the question you asked earlier about the comprehensiveness of studies in different systems – one of the really sad things to reflect on is, if you went back to 1989, when this was first started, and even 1992 when it was published, there was a lot more nature out there. Many species have gone extinct, many systems have become highly fragmented, many things have disappeared, become much less, much less vibrant, many communities are much more threatened and much less vibrant than they were back in 1989. This is a long time ago and biodiversity, natural systems and organisms in the field, in nature, have all been tremendously compromised since 1989. So that’s a very sad thing to reflect on, that many of these things will never be studied because they’re gone.
HS: Have you ever read this paper after it was published?
JG: Yes, but not recently.
HS: Do you remember in what context you read it? Did you read the whole paper or was it to look up specific things?
JG: Well, we relied heavily on what we did in that paper when I did that paper on the interaction between competition and predation, which was several years afterwards. So then we were really looking at the details. We’ve looked at it in various lab groups and so on, but I haven’t looked at it in a long time. And I think that’s a very good suggestion. That would be very, very fun to do with my lab group to go back and look at this paper and see what’s changed, what’s different, what we thought at that time. You know, that would be a very fun thing to look at, but I haven’t done that.
HS: Would you count this as one of your favourite pieces of work?
JG: Absolutely, yes. No question. As we discussed earlier, this really changed the trajectory of my career and was very influential. So yes, I have very warm feelings towards this accomplishment and towards the people that I shared it with.
HS: Do you still keep in touch with your co-authors on this paper? After this paper, did you work with them on other projects?
JG: Sadly, no. This was the only project we worked on together and they dispersed and went off to other careers. The only person I remained in touch with has been Laura Morrow, whom I greatly appreciate as a person and friend. I kept in touch with the others for some time afterwards, but have lost touch with them now, which is a shame.
HS: What would you say to a student who’s about to read this paper today? What should he or she take away from this paper? And would you add any caveats for them to keep in mind?
JG: I think you would have to read it in a historical context, in the context of what was known at that time, and what were we trying to answer, that may be well established now but that was not established then. What were the controversies at that time,what were the things we were trying to resolve, what was what were we doing, that was different than what had been done before? So, I think to think about things from the perspective of where we were coming from at that time. And then, why did it become influential, how did it change the way people were thinking about the problems we addressed and the subsequent direction of science since then? That would be an interesting thing to think about. But it’s very difficult to go back and think about how people saw things at that time and just the whole context. There was no Web of Science, there was no online access to papers, there was no email, there was no web, we didn’t have cell phones either. It’s hard to put yourself in that frame of mind, you know, how you would be thinking about problems and how you would be addressing and approaching things, when so many things were really different, were profoundly different about the way we were doing science,and certainly the way we were trying to make sense of scientific publications. Most importantly, a lot of the people reading this today weren’t even alive then, so it’s very hard to imagine how the world was before you are in it.
0 Comments