Statistical explanation[ edit ] A plot of the rank versus frequency for the first 10 million words in 30 Wikipedias dumps from October in a log-log scale. Although Zipf's Law holds for all languages, even non-natural ones like Esperanto the reason is still not well understood. Wentian Li has shown that in a document in which each character has been chosen randomly from a uniform distribution of all letters plus a space characterthe "words" with different lengths follow the macro-trend of the Zipf's law the more probable words are the shortest with equal probability.
He took a large class of well-behaved statistical distributions not only the normal distribution and expressed them in terms of rank. He then expanded each expression into a Taylor series. In every case Belevitch obtained the remarkable result that a first-order truncation of the series resulted in Zipf's law.
Further, a second-order truncation of the Taylor series resulted in Mandelbrot's law. It was originally derived to explain population versus rank in species by Yule, and applied to cities by Simon. Related laws[ edit ] A plot of word frequency in Wikipedia November 27, The plot is in log-log coordinates.
Most popular words are "the", "of" and "and", as expected. These lines correspond to three distinct parameterizations of the Zipf—Mandelbrot distribution, overall a broken power law with three segments: a head, middle, and tail. Indeed, Zipf's law is sometimes synonymous with "zeta distribution", since probability distributions are sometimes called "laws".
This distribution is sometimes called the Zipfian distribution. A generalization of Zipf's law is the Zipf—Mandelbrot lawproposed by Benoit Mandelbrotwhose frequencies are: f.