In order to ground psycho- and neurolinguistic work in linguistic theories, one needs confidence that these theories rest on sound empirical grounds. However, there is a long-standing concern that is periodically revived in the language science community that severe methodological problems plague linguistic theory, leading to its increasing isolation and irrelevance within the cognitive sciences. A reconciliation would only be possible, under that view, if syntacticians radically reformed their research practices and adopted research methods that more closely approximate the practices of other psychological sciences. In the particular domain of syntax, these methods have become known by the name of Experimental Syntax.
My collaborators and I have been engaged in an ongoing long-term project aiming to elucidate (i) how pervasive the methodological and empirical problems in linguistic theory suggested by critics actually are and (ii) how Experimental Syntax can otherwise be used to advance the goals of linguistic theory.
To that end, we have systematically compared large and representative corpora of acceptability judgments culled from textbooks and journal articles with acceptability judgments elicited from large samples of naïve participants, following the best practices of Experimental Syntax (Sprouse & Almeida, 2012; Sprouse, Schütze & Almeida, 2013). The data patterns reported by linguists were largely replicated (conservative estimates between 95%-98%). We have directly engaged the critical scholarship with these findings (Sprouse & Almeida, 2013ab [LCP, Cambridge Handbook of Biolinguistics], 2017 [BBS]), which has spurred replication attempts by other groups (Mahowald et al., 2016), resulting by and large in the replication of our original findings. It is interesting to note that this work predated by a few years the large scale replication attempts in psychology that have recently emerged due to concerns about a “replication crisis” in the field.
Notwithstanding that traditional generative syntax seems to perform at least as well, if not better, than even the most rigorous areas of psychological research in its ability to produce reliable empirical generalizations, we have continued to explore how Experimental Syntax could serve as a useful tool for theoretical linguists (Sprouse & Almeida, 2013, Cambridge Handbook of Biolinguistics). In particular, we clarified important statistical issues related to acceptability judgment experiments (Sprouse & Almeida, 2017, Glossa), and provided novel empirical findings that could shed new light on ongoing theoretical debates, like the source of syntactic island phenomena (Almeida, 2014; Tucker, Idrissi, Sprouse & Almeida, accepted).