Since the term crowdsourcing was coined in 2005, we have witnessed a surge in the adoption of the crowdsourcing paradigm. Crowdsourcing solutions are highly sought-after to solve problems that
require human intelligence at a large scale. In the last decade there have been numerous applications of crowdsourcing spanning several domains in both research and for practical benefits across disciplines
(from sociology to computer science). In the realm of research practice, crowdsourcing has unmistakably broken the barriers of qualitative and quantitative studies by providing a means to scale-up previously constrained laboratory studies and controlled experiments. Today, one can easily build ground truths for evaluation, access potential participants around the clock with diverse demographics at will, and all within an unprecedentedly short amount of time. This also comes with a number of challenges related to lack of control on research subjects and to data quality.