Articles

How to do great research?

Ekaterina (Katia) Damer
|August 28, 2018

Join us on our mission towards trustworthy data

It's becoming more difficult than ever to disentangle fact from fiction. Biases and motivated reasoning often lead us astray, preventing us from making informed and balanced decisions. This is why we at Prolific, as researchers ourselves, think that it's important to collect data to test our intuitions. This will help us and our users conduct the best research we possibly can.

We envision a world where the insights gained from research help improve society for the better, whether by informing social or organisational policies, by helping researchers develop new drugs and treatments, or by bringing about technological advances.

In line with the above thoughts, we're thrilled to announce our mission statement:

Prolific empowers great research. 🚀  We enable fast, reliable, and high quality data collection by connecting diverse people around the world. Our mission is to make trustworthy data more accessible in order to help improve human knowledge and decision-making.

We want to supercharge the speed at which researchers can collect high quality data, so they can answer pressing research questions. For example, such questions could be: How can individuals, companies, and institutions become more productive and creative? How can one increase environmentally sustainable practices and behaviours? How can we all become healthier and happier? Data can help answer these questions, and we want to help you gain access to quality data.

How to do research well?

In the coming months, we'll be launching a campaign to help you learn how to do great research. It doesn't matter whether you're an undergraduate student, a PhD student,  a postdoc, or a seasoned professor – there's much to be learned either way!

🥁 We're delighted to announce that we've put together a Best Practice Guide, which will help you with different stages of the research process. As part of this campaign, we'll point you to various super useful resources and we'll share our advice on how to make the most of Prolific.

Our Best Practice Guide covers the following phases of the research process:
1. Phase 1: What questions do you need to ask yourself before designing a study? (Hint: It really matters whether you're doing exploratory or confirmatory research. Plus, do think about your target population and required sample size in advance.)
2. Phase 2: What should you keep in mind while designing your study? (Hint: Be mindful about what reward you set for your participants.)
3. Phase 3 How to best pilot and launch your study? (Note: There are good and bad reasons to reject participants' submissions, so try to have fair rejection criteria.)
4. Phase 4: How to best analyse your data? (While this is normally beyond Prolific's scope, our team does have significant expertise in this area, so please check this section out.)
5. Phase 5: What to do after data collection? Or, how to best iterate the research cycle? (Sneak peek: We think it's best to aim for converging evidence and to build solid theories.)
6. Phase ∞: How can we ensure valid and reliable science? (Hint: There's no one easier to fool than yourself when doing research. Plus, we'll talk about the manifesto for reproducible science.)

We hope that you'll draw value out of our Best Practice Guide! As always, please don’t hesitate to ask us questions you may have or to provide any feedback or comments!

Challenges of online research

There are tons of challenges when collecting your data online from diverse people around the world.

Traditional solutions like panel companies don't quite do the trick anymore: They're unnecessarily expensive, slow, and non-transparent. Even sites like MTurk, which have paved the way for rapid online data collection a decade ago, are not fit for purpose anymore. It now seems uncontroversial that there are issues with shady MTurk workers, who conceal or distort their identities and provide false information. In the next 6 months, we'll juxtapose various sites and solutions for participant recruitment, and discuss their pros and cons.

In line with the above, data quality has come to the fore-front as one of the most important issues in online data collection today. As we have written previously, there's a lot that you as a researcher can do to improve your data quality. Because this topic is so fundamental to online research, we plan to sketch out in more detail how we're planning to improve data quality in the future.

How can you, as a researcher, find niche and diverse samples online? Is there any way at all to get representative samples quickly and at a low cost? We'll be discussing all these questions as part of our campaign. If you have any thoughts or feedback on any of this, please don't hesitate to let us know.

Introducing: Junior Researcher Grants

In order to empower great research, we want to give more junior researchers an opportunity to collect quality data. Next month, we'll be opening applications for our brand new Junior Researcher Grant Scheme: We'll be giving away a total of US$5,000 in prize money to junior researchers (that is, current grad students / PhD students / postdocs), so they can collect quality data via Prolific. The application process will be simple, although we'll place special emphasis on the adherence to open science practices. For example, one requirement for being awarded a prize will be that you preregister your research. More details to follow soon – stay tuned!

Introducing: Curious at Prolific (or, a look behind the scenes)

I've met quite a few people (both online and offline), who've thought that Prolific was a large company or even corporation. This couldn't be further from the truth: We're a small (but fast-growing) startup, with currently a team of 9 people. In the coming months, we would like to give you a sense of what it's like to work at Prolific, who the people are making it happen, and what Prolific's goals and ambitions are.

A look behind the scenes wouldn't be complete without glimpse into who our researchers and participants are: We'll be interviewing various Prolific users, sharing case studies of challenging experiments run via Prolific, interviewing some thought leaders in science, and showing you some of the faces of the people providing the data, our participants.

Credit where it's due

Here's to the people who are making the magic happen! Hannah Lettmann has been leading all of the above efforts – putting together the best practice guide, organising interviews, drafting educational posts, and managing social media.

Many others in our team – Data Analyst Jim Lumsden, Research Consultant Zak Evernden, Participant Support Manager Kendra Meyer, COO Jim Moodie, and CTO Phelim Bradley – have also made great contributions to the content we'll be sharing with you very soon. Last but not least, thanks go to our frontend developer Andrew Cox, who has developed Prolific's beautiful user interface, and our backend developer Stephen O'Malley, who has made Prolific's backend run so smoothly and safely.

Come discuss this blog post with our community!