Kicking off 2019 with Stanford's Gregg Sparkman

Did you know that cheese can have a larger carbon footprint than chicken or fish? If not, you have company. I didn't either. Experts now widely agree that climate change is real and that its unusually fast progression is caused by human behavior. Undoubtedly, it is one of the most pressing challenges for humanity today.

I'd like to kick off 2019 with an interview I've had with Gregg Sparkman, PhD, about a range of gripping questions around climate change, psychology research, and scientific progress.

Together, Gregg and I explored: What can we best do to counter climate change from a behavioural point of view? What are "dynamic norms" and how can they help deal with pressing social problems? Should science be based on trust? Is Mturk fit for research purpose? Are registered reports revolutionising the scientific process? Without further ado... meet Gregg! 🙂

Katia: Gregg, you’re currently a postdoc in psychology at Stanford University. After your BA in psychology and anthropology at Berkeley, you’ve completed your PhD at Stanford last year (congrats!). What’s your research focus?

Gregg: The bulk of my research looks at how people respond to learning that society’s norms are changing over time. In particular, I’m interested in how people’s behavior changes when they learn that others are changing. For example, there is a social norm around eating meat, but it turns out that levels of meat consumption have decreased in the past few decades. I’m interested in understanding how people respond to such information about norms changing.

K: You recently co-wrote an article in Slate arguing that people’s personal choices do make a difference for climate change. In other words, reducing one’s carbon footprint still matters. Why is that?

G: Some people are aware of the direct impact of their actions, but many aren’t. Consumer behavior makes up quite significant proportion of the causes of carbon emissions, so it’s not to be underestimated. In addition to this, there’s also a substantial social impact of taking action. Essentially, if we take action, then that increases the odds of other people taking action. While we definitely do need governments and companies to change their practices and policies, we absolutely also need consumers to take a stance. When consumers signal to industry what’s important to them through their behavior, then change will happen much faster and we’ll have a much better chance to cope with the effects of climate change effectively.

K: What are some of the most effective things people can practically do to reduce their carbon footprints? To my surprise, I recently read that having fewer children is perhaps the most effective way to reduce your carbon footprint.

G: This really does depend on what you’re personally willing to do. Having fewer children is one option, and adoption is something to keep in mind as an alternative to having your own child. Aside from that, I’d say that there are three key areas.

One area is travel. Flying is a major contributor towards carbon emissions. Personally, I avoid flying if I can, and have managed to do so since about 2010. For conferences and the like, I often take the train or help organize carpools.

A second area is energy consumption (in terms of home energy). People often don’t know this, but in the United States you can opt in to a 100% renewable energy plan. Doing so, you pay only about 10% more than you’d pay otherwise. Usually that only works for electricity, but not gas (unfortunately, there’s no renewable option for gas).

The third area is food choices. Cutting back on meat and cheese can go a long way. I include cheese because it can have a larger carbon footprint than chicken or fish.

K: Wow, I did not know that cheese can have a larger footprint than chicken or fish! Now let’s talk about some of the experiments you report in the Slate article. For example, you report research on “dynamic norms”, which suggests that changing people’s perception of norms can change their behaviors, for example environmentally sustainable behaviors. How does this work? How can you change people’s norms? Don’t norms take a while to culturally emerge, or are they perhaps more malleable than I imagine?

G: We don’t change people’s norms as such, but rather their perceptions of social norms. People don’t typically have a true sense of what others are doing, they just have a perception of it, which is influenced by a lot of things. It’s influenced by actually observing people, but there’s a lot of research showing that it’s also influenced by the media and fiction that you read or watch. So, people’s sense of what other people do is fairly easy to influence; it’s flexible.

Our research added to the existing literature by talking about what direction of change there is in norms over time. That hadn’t really been examined before. We looked at this in the context of meat consumption. In the United States there has been a declining meat consumption, a little more than 10% since 2007 (per capita). I think almost all of that was actually beef specifically that went down.

In our studies, we shared information that many people had started to reduce their meat consumption. By making this information salient, people started thinking that it might be a good idea to reduce meat consumption for themselves. This effect was in part driven by people thinking “If someone’s willing to change for this, maybe it’s important”. It was also driven by people thinking “If things are changing now, they might continue to change and maybe many people will eat less meat in the future”. Essentially, they pictured a changed future norm and conformed to it like a current reality. We called this “preconformity”.

In a field study at a local cafe, we then found that learning about this information via a survey while people waited in line actually changed people’s food choices. We didn’t inform people that we were monitoring their food choices – we just gave them the survey and a coupon for a discount on lunch while they were in line. But, each coupon had an ID, so we could track what they ordered. At the end of the day, we tallied up who had ordered meat and who hadn’t.

People don’t typically have a true sense of what others are doing, they just have a perception of it, which is influenced by a lot of things. It’s influenced by actually observing people, but there’s a lot of research showing that it’s also influenced by the media and fiction that you read or watch. So, people’s sense of what other people do is fairly easy to influence; it’s flexible. – Gregg Sparkman, PhD

K: What a neat experimental design! This is a great example of how to do rigorous, ecologically valid field research to corroborate findings from the lab. Although now I’m wondering… not everyone might respond positively to learning that meat consumption is declining. There are many people who love eating meat, so was there any backlash at all? Or perhaps a better way of putting this is: Was there any evidence that some people weren’t receptive to learning about this kind of information?

There’s work showing that norms appeals can make people feel pressured, or annoyed, and that it can make them feel frustrated with whoever is delivering that appeal. We went to great lengths to avoid making this seem like an appeal to eat less meat. We told people that we were just doing consumer research and this was aided by the fact that we had a control group that was about Facebook use, so we were looking at a variety of topics and at no point did we ask them about their personal behavior. We just asked “Hey, there’s this trend going on, why do you think that is?”. Having them reflect on why other people might change, this increased the odds of behavior change. We always pre-test our materials to make sure that we’re able to avoid the detrimental effects of our interventions.

K: That’s actually a really good take-away for doing research in general – I think we often don’t pretest enough. How do you typically collect your data to investigate your research questions? Do you have any preferred way(s) of collecting data?

G: We usually start in a quick and fast and easy way by doing online surveys, and then do field studies after that. And when we do the online research, we often use Amazon Mechanical Turk, unless there is some other kind of convenience sample (for example, we might recruit people we know for pilot tests).

K: I believe you’ve seen the controversy around shady MTurk workers who use server farms to conceal their identity and get access to paying surveys. What do you think about these concerns?

G: Yeah, it’s a growing concern. We had one data set where, in our first sweep of analysis, we didn’t get the results we hypothesized. And then we went back using a coding scheme that was similar to the one used by TurkPrime to try to identify people who might be operating from abroad, or who might be in a click farm. Labor-wise what it entailed was going through the 1,200 participant responses and manually coding responses for poor or mediocre grammar and spelling. We then combined this with repeated geo location, which can be a symptom of using VPNs. We then pulled out about 16% of the sample that very obviously did not seem to be US participants. These responses were pretty dismal… some of them were really, really bad! And these are only the ones we know about…

At the end of the day, you have a really hard time figuring out: Is it people in the US who use VPNs to legitimately protect their information, or is it people abroad who want to circumvent the system? As researchers, should we be conservative and drop all of them, or should we do it case-by-case? I often end up doing the latter.

K: It sounds like a huge pain to have to tease out which MTurk participants are legitimate, and which aren’t. So what do you think: Could this be the end of MTurk? Would you consider moving to a different platform?

G: Yeah, for sure! It doesn’t seem like Amazon has much of a response to this. They didn’t come up with any great solutions to this. TurkPrime has created a list of workers that are most likely from clickfarms, but that’s a growing list and there’s no way they can keep a perfect list for that kind of thing. It will be really hard to keep up with the list. Even with really good survey designs, it’s still very hard to capture who is who. You could take a conservative approach and drop participants with the same geolocation, but then you could be dropping people who live in the same city block, which doesn’t seem ideal. So, none of those methods are perfect. I think a lot of people do not know much about these data quality problems on MTurk, so their data is suffering. Prolific looks like a great tool, its values very much resonate with me. I think I’ll try it out for some of my upcoming pilot studies.

I think a lot of people do not know much about these data quality problems on MTurk, so their data is suffering. Prolific looks like a great tool, its values very much resonate with me. I think I’ll try it out for some of my upcoming pilot studies. – Gregg Sparkman, PhD

K: Let’s switch gears a little. What do you love about doing science? Why is it important that we do research, why do you care about it?

G: Personally, I really like being a researcher and pursuing research questions for two reasons. One is to learn new things that we did not know before and the other is to make progress on contemporary social problems. My approach to research tries to address both of these. I pick social problems where we don’t yet know how to make progress on them, where we don’t have a solution that seems viable. I take the fact that it’s a social problem that still exists as a cue that there’s something else for us to learn in that context. In my case, the dynamic norms research actually came out of the meat consumption context. It wasn’t the other way around. My general course of research is to examine contexts where we don’t have obvious solutions and then develop new theory in order to help provide some positive change in that context.

K: That’s beautiful! By theorizing how to fix social problems you drive theory development. Is there anything that you really don’t like about current scientific practices, and want to see changed? For instance, how do you feel about preregistration?

G: I see the benefit of preregistration mostly in that it clarifies what’s a-priori and what’s not. People act as though this is easy, but I think it’s a difficult task, and I don’t think that people always receive proper training to actually do it well. So while I see the value in preregistration, I also think it’s important to remember that science, at a more foundational level, necessarily requires trust. If people were going to fake their data, there’s not much you could do to stop them. There’s some level at which we have to trust people, and I feel like we haven’t negotiated that space well enough yet. I could see the value in moving towards mandatory preregistration, but ultimately I don’t think that’s what’s going to solve our problems because we need a system where people are educated, trustworthy, and have incentives that make sense.

K: I think it’s debatable to what extent science should be based on trust: Science is so fundamental to many decisions that are being made, by governments for instance. If it’s just based on trust, then that’s quite a risky strategy that might lead us astray. Transparency is one way of dealing with this risk. If you make your code, your data, and research materials transparent, then fellow researchers can double-check what you’ve done; this allows for science to be self-correcting. Given that we (as a society) want to maximize our chances of spending our resources wisely, and given that we know that people are good at fooling themselves, it’s key we have some checks and balances in place when doing science.

G: I think that is a large concern: How easy it is to fool oneself. I think it’s important to ask: Why do we think that we can’t trust what scientists do? Part of the answer is that the incentives are currently off right now – you’re rewarded for findings, not for hard work. People talk about solutions to all kinds of problems in science, but we don’t seem to be talking enough about incentives. The other concern I have is that it’s actually really hard to do good science. It’s a skill. Even if you’re not malicious at all, you are able to completely fool yourself. I think learning about how to specify one’s research plan a-priori should be a standard course in everyone’s education either through preregistration or another method.

Even if you’re not malicious at all, you are able to completely fool yourself. I think learning about how to specify one’s research plan a-priori should be a standard course in everyone’s education either through preregistration or another method. – Gregg Sparkman, PhD

K: Have you heard of registered reports?

G: Is that where you keep the preregistration private?

K: Not quite, registered reports are all about submitting your research plan to a journal, getting feedback from reviewers before you collect your data, and then, if the research plan is deemed to be of high enough quality, you’re given in-principle acceptance, which means that your research is accepted for publication before you collect your data.

G: I think this especially helps on the meritocracy side of things. Your success doesn’t depend on the results of your research, but instead on how well you’re asking questions and testing them. I think that’s a more emotionally rewarding system to be in!

I also wonder what to do if you want to retract your project if you don’t like your own results. Is that even an option with registered reports? Let’s say you’re really invested into an idea, but then have some kind of manipulation check failure and then you (as the author) say: “I don’t want this to get published yet, but I still really like the core idea of this project. I don’t want this publication to be the representation of the idea because I can still improve the research design”. Is it possible to retract a submission in the registered reports format?

K: That’s a really interesting question! Chris Chambers has been one of the pioneers when it comes to registered reports. In the past couple of years, he’s convinced over 100 journals to implement registered reports. I'll ask him on Twitter what he thinks about retracting a registered report! 🙂

To finish, do you have any advice for budding psychology researchers? Given how messy the scientific process and how demanding the academic world can be, how to stay motivated?

G: The choice of a research context that involves social problems is also a choice I make because it’s emotionally rewarding and motivating. Conducting a research project takes a long time before you have a concrete output. So, you will need sufficient intrinsic motivation to keep going at it. I think that yoking into something that you care about in the “real world” is a great way of doing that. Whenever you do research that closely relates to the “real world”, there will be someone in applied settings who will love to hear about it. That experience is often one of the most rewarding. I think people often forget that science is a process. It’s not a process that you enact perfectly, and then it’s done. I don’t think that’s how anything works (laughs)! Science grows over time, it improves over time – it doesn’t come with perfect, capital T truths. It’s a process wherein you try to build a model of how the world works and we just get slightly better at making models that aren’t as bad as the last model. That’s our goal mostly! I hope people aren’t disillusioned about what science is.

Science grows over time, it improves over time – it doesn’t come with perfect, capital T truths. It’s a process wherein you try to build a model of how the world works and we just get slightly better at making models that aren’t as bad as the last model. I hope people aren’t disillusioned about what science is. – Gregg Sparkman, PhD

K: So then it’s in part about setting expectations, right? If you set realistic expectations about what you’re trying to achieve, then maybe science is less disillusioning or less frustrating.

G: Yes – science is an attempt at making progress on a question or topic, and that’s OK. It’s worth our time and effort even if it doesn’t yield precisely what you wanted or if you’re frustrated by what other people are passing off as science. It’s also a cultural process – by that I mean the idea that we have to get used to the knowledge base and to the values that it takes to conduct science.

K: Thanks ever so much for your time, Gregg!