When and Why Trump's Tweets Are Inappropriate, An Analysis

When Donald Trump tweets, the world listens. News organizations carry stories of his 140-character outbursts with analysis of his motivation and the implications. Some tweets are mundane and others break nearly every convention of Presidential behavior.

But what do people find appropriate and inappropriate? We decided to find out.

Data Collection

First, we downloaded all of the tweets from Trump's Twitter account posted between January 1, 2016 and February 5, 2017. This time period covered his candidacy and early Presidency. We eliminated retweets and quoted tweets so we only considered posts written by someone using the account. This gave us 3,666 total tweets.

Next, we asked Americans to rate the tweets. We had a pool of about 175 people. Each tweet was rated by 10 self-described liberals and 10 self-described conservatives, randomly selected from the pool. Raters were asked if the tweet was "Appropriate", "Neutral", or "Inappropriate". They were explicitly told not to rate based on whether they agreed politically with the statement but rather to rate if it was an appropriate statement that a Presidential candidate or the President should be making.

Appropriate tweets get a score of 1, inappropriate get a score of -1, and neutral get a score of 0. We averaged the rating of all 20 raters to get a score for each tweet. An average of 0.9 would mean 19 people labeled the tweet "appropriate" (1) and 1 person who labeled it "inappropriate" (-1). An average score of -1 would mean all 20 people labeled the tweet "inappropriate".

Average Score

The average tweet scored a 0.51, halfway between neutral and appropriate. 780 (23%) of tweets had an average negative score, putting them on the inappropriate end of the scale. However, there are a lot more interesting ways to break this down.

By Political Leaning

Not surprisingly, liberals and conservatives differ in what they find appropriate. The average rating from liberals 0.38 while conservatives gave an average of 0.64. From liberals, the average score of 1,064 (29%) of the tweets was negative, indicating they were inappropriate.

We will continue this political breakdown for other differences below.

By Platform

Conventional wisdom holds that Trump himself posts from an Android phone while his staff posts from an iPhone. Those were the two most popular devices used to post to his account, but we found a few others. To the left is every platform used and the number of tweets posted from it.

But let's stick to just comparing Android and iPhone. Take a look at the averages in this chart, broken down by platform and rater's political preference.

First, everyone thinks the iPhone staff tweets are more appropriate than the ones Trump writes himself on Android. There were dramatic differences in liberal and conservative views. Liberals give Trump's own tweets an average of only 0.1. Nearly half (49%) of Trump's tweets averaged neutral (0) or inappropriate (<0) when rated by liberals. On the other hand, conservatives give Trump much higher marks and the staff tweets have quite a high rating.

These differences in appropriateness from Trump vs. Staff play out in other ways.

The Worst of the Worst

There are a lot of tweets (1,297) that all of our raters agreed were neutral to appropriate (i.e. no one said they were "inappropriate"); 647 tweets were ranked totally appropriate by all 20 raters. Most of these came from iPhones, but some (118 of the 647, or 18%) came from Trump himself. These were pretty typical campaign tweets for the most part, e.g. "It was so great being in Nebraska last week Today is the big day—get out and vote! "

However, there were some tweets that were considered inappropriate by liberals and conservatives alike. Trump's tweet threatening to "spill the beans" on Ted Cruz's wife received the lowest score; it was ranked inappropriate by everyone except one conservative.

A few others received very low average ratings of -0.85, including "Explain how the women on The View, which is a total disaster since the great Barbara Walters left, ever got their jobs. @abc is wasting time" and "Wall Street paid for ad is a fraud, just like Crooked Hillary! Their main line had nothing to do with women, and they knew it. Apologize?"

There are 37 tweets that liberals unanimously deemed inappropriate. These include the tweets mentioned above, as well as criticisms of Megyn Kelly, John McCain, Elizabeth Warren (particularly calling her "Pocahontas"), and Mitt Romney: "Mitt Romney, who was one of the dumbest and worst candidates in the history of Republican politics, is now pushing me on tax returns. Dope!"

By Time

If we look overall at ratings of appropriateness compared to the time of day when the tweets were posted, at first it looks like there is a strong relationship. (Note: this graph shows the overall average rating, but liberals and conservatives have basically the same shape line, just shifted up for conservatives and down for liberals)

But that's not the whole story. It turns out that Trump tends to post by himself early in the morning while the staff posts mostly during the day. You can see that in this chart that compares Trump's Android posts (blue) vs iPhone posts (red)—they post at very different times of day.

The relationship we observe between appropriateness ratings and time of day is actually capturing ratings of Trump's tweets vs. staff tweets. Because Trump posts more in the morning, this explains why the morning tweets are considered more inappropriate—Trump's tweets overall are considered less appropriate. As Trump's fraction of tweets in an hour declines, the appropriateness ratings increase.

We can see this statistically. The correlation between appropriateness and the percentage of tweets in an hour from Trump is -0.86. That is a very strong correlation. It means the greater the percentage tweets from Trump and his Android phone in an hour, the less appropriate people think that hour's tweets are.

There is some relationship between time and appropriateness if we control for the platform. The correlation between ratings and the time of day when an Android tweet was posted is about 0.3. It suggests that people find Trump's early morning tweets less appropriate than what he posts later in the day. We can confirm this with a t-test. Android tweets posted between 5am and noon (avg rating 0.28) score significantly lower (p<0.01) than android tweets posted from noon to midnight (avg 0.38). Trump's really early morning tweets are the most inappropriate. Tweets posted between 5am and 7am, average a negative rating (-0.1) from liberals and 0.4 from conservatives.

The Shabbat

There is a theory that Trump posts crazier stuff from sundown Friday to sundown Saturday when Ivanka and her husband are not working because of shabbat. Slate did a "scientific" study of this, asking a few people around the office if they thought his tweets were crazier than a few other tweets, and concluded they were.

We're doing some real science here with a much bigger sample. We took Trump's android tweets from shabbat and the android tweets from the rest of the week and compared the appropriateness ratings. There was no difference, statistical or observable, of ratings for shabbat tweets vs others—people's scores were exactly the same.

I suspect the impression that shabbat tweets are crazier is that Trump posts at night and on Saturday morning and there may be fewer staff tweets on a Saturday to balance the impression.

The Words

Next, we looked at the kinds of words used in these tweets. We separated them into Fully Appropriate (received scores of appropriate from all raters—647 tweets, 82% of which were non-Android) and Inappropriate (had an average negative score—780 tweets, 62% of which were Android posts). We ran these through LIWC (Pennebaker 2001), a psycholinguistic text analysis tool that tells us what kind of words were used in each sample.

There is a language style difference between the two categories. The appropriate tweets use a lot more punctuation, six times the rate seen in the inappropriate tweets. Appropriate tweets also use more big words, 40% more words over six letters long.

The personal focus varies. Inappropriate tweets use "he/she" over 16 times more often than appropriate tweets. Appropriate tweets use "we" 3.3 times as often and "you" more than 17 times as often as inappropriate tweets.

Emotional content also differs, inappropriate tweets are more negative. The inappropriate tweets used words related to negative emotions more than 8 times as often as the appropriate tweets; words associated with anger 6.6 times as frequently; and words associated with anxiety 7.4 times as frequently. Appropriate tweets use words related to positive emotions at 3.3 times the rate of the inappropriate tweets.

Conclusions

Trump is changing what we expect in public statements from our leaders. We looked at appropriate vs. inappropriate tweets. Remember, inappropriateness does not mean that the rater disagreed with the politics expressed in a tweet; it means they believe it is something that should not be posted by any person in Trump's position.

Liberals and conservatives differ in what they feel is appropriate from a President or Presidential Candidate (nothing surprising there if you have been paying attention). However, they agree that Trump posts inappropriate things, far more inappropriate than what his staff posts. His early morning tweets are especially disliked. There are also marked differences in the language used between appropriate and inappropriate tweets, with appropriate tweets more sophisticated, more positive, and more likely to come from staffers rather than from Trump himself.

It will be interesting to see how these feelings change going forward. If we ask these same people to rate these same tweets in a year, will Trump's tweeting behavior have normalized? Will these posts appear more appropriate because the standards have changed? Or will there be greater pushback against perceived inappropriate behavior? We will check back in to see.