Morphological observation and measurements of endocasts have played a vital role in research on the evolution of the human brain. However, endocasts have never been used to investigate how the human brain has evolved since the Neolithic period. We investigated the evolution of the human brain during the Holocene by comparing virtual endocasts from Beiqian site (a Neolithic Chinese site) and a sample of Chinese modern-day humans. Standardized measurements and indices were taken to provide quantification of the overall endocast shape, including the length, breadth, height, frontal breadth, and the ratio of frontal breadth to breadth, as well as the cranial capacity. We found that the height of the endocasts and cranial capacity have decreased between our two samples, whereas the frontal breadth and sexual dimorphism have increased. We argue that these changes can be caused by random genetic mutation and epigenetic change in response to changes in the environment.