Topic modelling is a technique for seeing larger patterns in a (usually large) corpus of documents. The original impetus for this entire project was curiosity about what applying this technique to the full run of a general, academic journal in Jewish studies might reveal about the state of the field, or at least the state of the field through the eyes of its editors. It is technically an “unsupervised” method, in that the computer determines through its algorithms which words appear together more frequently than others without human guidance. In reality, there is quite a bit of human tweaking that goes into making these models, beginning with determining which words should remain as significant and which should be eliminated (or “stopworded”) as noise.
For these models, we used a technique used as Latent Direchlet Allocation (LDA). There is an element of probability in these models, so even using the same data will often present somewhat different (but widely overlapping) lists of topics and their components. We experimented (as one must) with the various parameters that go into making such a model (especially the number of topics to use, which must be set from the outset, the number of words to use for each topic, and the number of “passes”) until we got a set of topics that individually looked relatively coherent and collectively were diverse. Since AJS Review does not publish many articles each year, we divided our analyses into (approximately) five year blocks, ending with the last year of data available to us, in 2014.
There are different ways for visualizing topics, but one of the more interesting and interactive ways comes out of a package known as pyLDAvis (cf. this description of topic model visualizations), which runs from the topic model package that we used (gensim). These visualizations can be accessed through the links below.
