Trends in Classics: What the Critics are Saying (and How the Machines are Changing It)

We’re back again with the fifth installment of our series exploring trends in classical scholarship. Last week I sent a computer to do an undergrad’s job, and had it tell me which sentences in each of the ~8000 (English) Bryn Mawr Reviews were complimentary and which ones were critical. This time around, I’ve pulled out all of the sentences that fall into those two categories, and I’m going to take a (somewhat) closer look at them. What things do we most frequently laud? Where do we find fault? Is there anything here that I can blame on the internet? As always, the answers are below the jump. 

Below there are two word clouds, one each for sentences of praise and blame, as flagged by my homemade sentiment analysis program. For obvious reasons, these and all of the other images below exclude those features (words and combinations thereof) that the program uses to identify praise and blame, as well as the usual list of stopwords. As you can see…

Word cloud of praise and blame, highlighting colons

Again? Sigh.

Really? This again? Apparently it isn’t just titles where we can’t stop ourselves from using colons — it’s any time we write anything that we think someone might read. “Colons” includes both the regular and semi-variety because if I plotted them separately, nothing else would fit. All told, for 8,000 reviews we used almost 12,000 (semi-)colons in sentences where we praise the book, and another 4,500 in sentences that criticize it. If raw numbers are your thing, 1 in every 3.7 sentences of praise has a colon or semicolon, and the same is true for 1 in every 2.6 sentences of criticism. Yeesh. I really need to believe that classicists don’t use colons in one sentence out of three as a general rule, so I looked at a few dozen randomly drawn critical sentences which include a colon of some persuasion. Most of them fell into three categories. Lists of typos and minor errors separated by semicolons account for about half of the total (phew!).(1) The rest are fairly evenly divided between criticisms that include the title of a book (2) and attempts to moderate or ‘soften’ criticism that might otherwise seem harsh by extending the sentence.(3)

Next, I’ve given you word clouds for the 150 words most commonly found in sentences of praise and criticism respectively. Of necessity, I’ve added both colons and semi-colons to the list of stopwords, so they won’t appear. The word clouds are color-coded to reflect how ‘trendy’ they are.(4)

Word cloud showing words of praise; top 3 are Green, book, one

Apparently Hellenists find more to like than Romanists?

By and large, this is about what we might expect, but there are a couple of points worth noting. One, just to the left of ‘book’, you can see that ‘p.’ is getting less frequent. Apparently we are growing less likely to point to a specific page as containing something we like. Conversely, ‘chapter’ and ‘section’ are becoming more common [Ed’s note: can this relate to the (anecdotally) increasing frequency of the “book précis” as part of a BMR? Chapter-level criticisms seem less likely than chapter-level praise … NB that the author’s response to my suggestion was ‘needs more looking into’]. Those with the eyes of an eagle might also notice a fortuitous technical glitch. Hidden away are a very small purple the and an equally tiny, red ‘the. Both of those are artifacts of my computer not knowing where to put a word-break, and deciding that those are unique words and not the same as  the the which gets filtered out as a stopword. Why is it fortuitous? Because it tells us something that we wouldn’t otherwise know: we put direct quotes in sentences of praise so frequently that both the and “the are both in the 150 most common tokens in those sentences. And this only works if the quote begins with ‘the’! And only when my word tokenizer chokes! And it isn’t even homemade!

One final point: near the bottom, dead center, running vertically, you’ll notice that we have grown less likely to single out an index for praise. This, I suspect, is related to improvements in technology. Either indexing software has gotten sufficiently good and ubiquitous that all indexes are uniformly pretty OK, or, more likely, a certain advertising behemoth has conquered the world to such a degree that we no longer rely on a book’s index for information retrieval, so none of us [Ed’s note: few of us?] care anymore.

Moving on to blame!

Word cloud of most common words of blame in classics reviews. Top three are greek, book, one

Or maybe Hellenists are just more numerous…

Here too there are few major surprises, but some interesting trends. First, if you look just to the left of the blue ‘even’, you’ll see an almost impossibly small, blue ‘n.So the number of times that a book review includes criticism of a specific footnote is on the wane. Which, on the whole, is good—let’s try not to be too much of a caricature here. Of course, the fact that it appears here at all means that we still do it enough for it to be one of 150 most common tokens in these sentences. [Ed’s note: typos? Also, I think it’s interesting that the exact same three words are the most common in praise and blame — but I guess that just reflects what gets published.]

Likewise, if we look at the outer edge at around one o’clock, mention of the word ‘bibliography’ in our critical comments is frequent and growing more frequent. We actually seem to be in the midst of a major shift in how we evaluate bibliographies. From 1999-2002, positive comments were more likely to contain the word ‘bibliography’ than negative comments (it appeared in 2.2% and 1.6% of positive and negative comments respectively), but in the four most recent years (2011-2014) that has flipped (2.36% and 2.41%).(5) Digging a little deeper, the shift is primarily driven by a single idea. From 1999-2001, nearly two thirds of our positive comments about bibliographies lauded them for being ‘helpful’ or ‘useful. For 2011-2014, that has dropped to less than half.(6) This feels like another symptom of technological change. A related book’s bibliography used to be hands-down the easiest way to find relevant scholarship. The ubiquity of information-retrieval technology means that often isn’t the case now, and even when it is, it isn’t likely to be our first choice because it is less convenient [Ed’s note: and out-of-date within a year].(6) In other words, now that we don’t ‘use’ bibliographies as a research tool, it’s gotten harder to say nice things about them. The main nice thing we can say is that they’re long.

Speaking of length, maybe I’ll wrap up this by just dropping one more graph here.

Graph showing relationship between length and praise of book.

All else being equal, each 100 additional pages gets you an increase of about 5% of the average score. Relationship significant at… p=0.000001? Lol.

For those of you who haven’t taken statistics in a while, significance is usually calculated at p =0.05, or that this could happen by chance one time in 20. If there were no real relationship between number of pages and praise, these results would happen by chance about one time in a million.


  1. E.g. “I noted the following printing errors: on p. xxii, “over the Crete” should read “over to Crete”; 14, “The words <that> concluded”; 14, “triangular” is misspelt; 44, “quick<l>y”; 121, “O ‘Con<n>or”; 174, “suitors< ‘>”; 190, “Klytaimnestras.
  2. E.g. “A glance at the chapter on population and demography reveals the omission of W. Suder, <i>Census populi: bibliographie de la démographie de l’antiquité romaine</i> (Bonn 1990), the most important bibliography on the subject.
  3. E.g. “7 [Ed’s note: we’re not sure what this means either] suffers a bit (I think) from its failure to deal adequately with what has recently come to be known as “endoxic method; his two first chapters, on the other hand, which deal with the life and works of Aristotle are sufficiently rich and up-to-date to suffice for the needs of most readers, especially beginners.
  4. The calculation to determine trendiness is difficult to explain succinctly, but intuitive. It compares the frequency with which a given word appears in a sentence (of praise or criticism) in the four-year periods from 1999-2002 and 2011-2014. E.g., ‘analysis’ appeared in 176 of 7445 (2.4% or all sentences) sentences of praise for the earlier period and 270 of 9250 (2.9%) for the later period, which works out to an increase of ~19%. It may be worthy of note that I originally intended to compare the frequency with which these words appeared in a given review, but reviewers actually became less ‘judgmental’ over the same time frame. When the millennium was new, reviewers expressed 1.7 criticisms and 6.2 compliments per review on average. That has fallen to 1.03 and 4.3 respectively.
  5. If those numbers seem too low to worry about, bear in mind that no single word that carries any real meaning (i.e., excluding normal stopwords + ‘greek’,’book’,’would’,’one’ and ‘also’) appears in more than 6% of positive or negative comments.
  6. This is a personal impression based on how my habits have changed. If you have your own opinion, feel free to drop it in the comments, and we’ll crowdsource an explanation.

~J for our anonymous author.


Creative Commons License
Trends in Classics: What the Critics are Saying (and How the Machines are Changing It) by is licensed under a Creative Commons Attribution 4.0 International License.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s