Word frequency distributions /
R. Harald Baayen
- Netherlands : Springer, c2001
- 333 pages ; 23 cm.
Includes bibliographical references and index.
1. Word Frequencies -- 2. Non-parametric models -- 3. Parametric models -- 4. Mixture distributions -- 5. The Randomness Assumption -- 6. Examples of Applications -- A. List of Symbols -- B. Solutions of the exercises -- C. Software -- D. Data sets -- Bibliography -- Index.
This book is a comprehensive introduction to the statistical analysis of word frequency distributions, intended for computational linguists, corpus linguists, psycholinguists, and researchers in the field of quantitative stylistics. Word frequency distributions are characterized by very large numbers of rare words. This property leads to strange phenomena such as mean frequencies that systematically change as the number of observations is increased, relative frequencies that even in large samples are not fully reliable estimators of population probabilities, and model parameters that vary with text or corpus size.