Working Papers

Schnier J., Dahlander L., Raasch C.: [A Study on Evaluation Errors].

Reject & Resubmit at Management Science, Job Market Paper

Managers often struggle to evaluate novel ideas, which by their nature have no proven commercial and technological value yet. Their evaluations can be overly optimistic or overly pessimistic, either over- or underestimating the idea’s actual performance. How do managers respond to these evaluation errors? We answer this question using unique panel data from the idea management system of a large firm in the aviation industry. With a difference-in-differences framework, we find that managers reject more ideas following overestimation errors but are unresponsive to underestimation errors. We explain this finding with strategic nay-saying: fearing the career costs of additional errors, managers preemptively reject ideas because their value remains unknown. We also consider alternative accounts of managers’ error responses, including learning from errors and intensified search for information, but find inconsistent evidence. Our study uncovers a new source of managerial conservatism rooted in an asymmetry in the observability of false negatives and positives.

Schnier J., Raasch C., Schweisfurth T.: [A Study on Evaluators' Own Idea Generation].

Minor Revision at Management Science

Organizations rely on their employees to produce many high-quality ideas for the improvement of products, processes, and strategies. Evaluators such as managers and technical experts must assess these ideas, while also being expected to contribute ideas of their own. We investigate this task duality, asking how idea evaluation activities affect evaluators’ ideation performance. While much of the creativity and innovation literature points to a crowding-out effect, some theoretical arguments support idea crowding-in. Using data from a multinational firm’s idea management system and a difference-in-differences framework, we find robust evidence of substantial idea crowding-in: evaluators’ likelihood of idea generation increases by 205% on the day they evaluate ideas, and remains elevated for the following two weeks. Extensive analyses support two mechanisms: idea evaluation shifts evaluators’ attention to ideation and creates new opportunities for knowledge recombination. The ideas that evaluators generate post-evaluation are 19% more valuable (but not more novel) than those they generate at other times. We contribute by identifying a hitherto overlooked effect of idea evaluation: triggering idea generation among evaluators.

Schnier J., Raasch C., Patat F.: Shifting Focus: How Panel Discussion Hurts Prediction Accuracy in Idea Evaluation.

A central challenge in science and innovation is to predict the performance of ideas that are inherently novel and uncertain. To master this challenge, many organizations rely on groups of experts, or evaluation panels, who come together to discuss and evaluate ideas. The purported advantage of panel discussions lies in the pooling of expertise. Yet, it is unclear whether panel discussions deliver on this promise. We examine the effect of panel discussions on evaluation accuracy using peer review data from the European Southern Observatory (ESO), Europe’s premier organization in astronomy that operates world-class telescopes for use by observational astronomers. Reviewers are required to grade proposals both before and after a discussion with other reviewers. We find that pre-discussion grades are 41 % more predictive of proposals’ future research output than post-discussion grades. We attribute this prediction accuracy gap to discussions shifting evaluators’ attention away from proposals’ scientific merit toward the more salient and accessible proposal attribute of implementation costs. Consistent with this interpretation, we find high-cost (yet high-gain, or `breakthrough') proposals to suffer most from discussion. We contribute by integrating the literature on group decision-making with that on idea evaluation.

Schnier J., Tartari V.: Strategic Idea Revival: How Creators Leverage Evaluator Turnover to Resubmit Rejected Ideas.

Idea rejection is a common yet often overlooked aspect of the creative process. While prior research emphasizes the benefits of revising ideas after rejection, we propose an alternative strategy: strategic idea revival. Rather than modifying their ideas, creators may choose to resubmit rejected proposals when the evaluators responsible for rejection have departed. Using data from the Novo Nordisk Foundation’s proposal review process, we find that applicants are more likely to resubmit rejected ideas when a greater share of intellectually close evaluators have left the panel. Moreover, evaluator turnover increases the likelihood of resubmitting “boomerang” ideas—those nearly identical to the previously rejected idea. Strategic idea revival yields selection rates comparable to idea revision, challenging the dominant view that creators should pivot after rejection.

Working papers available upon request.