Why Thanksgiving Flying Isn’t All That Bad
Siddhant Shandilya, Benjamin Livingston, Amogh Mishra, and Mohit Gulla
December 10, 2019
library(ggplot2)
library(dplyr)
library(ggthemes)
library(forcats)
library(ggpubr)
library(lubridate)
library(tidyr)
library(stringr)
library(reshape2)
library(usmap)
# remember to unzip data_clean contents to main directory!
thanksgiving = read.csv('../thanksgiving.csv')
alldates = read.csv('../alldates.csv')I. Introduction
“Thanksgiving is the worst time to travel!”
Any American has heard this adage a billion times - but really, what does it mean?
Americans live in constant fear of holiday travel, most without any real understanding of how bad it actually is and when the best times to travel actually are. This is particularly true for Thanksgiving - we’re vaguely told to avoid Wednesday afternoon and Sunday, without any real understanding of the landscape.
The authors of this study experienced the volatility of Thanksgiving travel firsthand in 2019. During a Wednesday train trip from New York to Philadelphia, the American of the group told the others, “Penn Station is going to be a mob scene.”
Strangely enough, though, Penn Station was calm when the group departed around 1 p.m. To the group’s eyes, it looked no different than any other day at 1 p.m.
Even stranger, 45 minutes later, Penn Station had become an unmitigated disaster. Just like that.
Thanksgiving travel is a fickle beast, and perhaps aside from the 405, no form of travel feels more vulnerable than air travel.
Air travel is an especially interesting entity to tackle, because there are so many options. If you’re driving from Philadelphia to D.C., you take I-95. If you’re taking the train from New York to Boston, you take Amtrak. But when you’re choosing your flights, there are so many more options in terms of airlines, local airports, and connecting airports, on top of the time old-question of when to leave to “beat the rush.”
We aim to answer all of these questions specifically for the Thanksgiving holiday, and help fliers make decisions about the best domestic airlines to use during the period.
Important Notes
What Period(s) We’re Dealing With
All of our analyses will deal with the five-day “Thanksgiving Holiday” period consisting of the Wednesday before Thanksgiving through the Sunday after Thanksgiving for the years 2015-2018 (prior to 2015, some of the recent airline mergers had not yet been completed, so we will leave those years out). We will also occasionally refer to “Pre-Thanksgiving”, the 20 day period before this period (ending with the Tuesday before Thanksgiving).
What Is A “Delay”?
When we say “delay”, we mean arrival delay, not departure delay. This is based on the assumption that travelers are most concerned with when they get where they’re going.
Our delay statistics are calculated as follows: if a flight wasn’t delayed by a minute or more, its delay value is considered to be zero (we won’t award flights for arriving a few minutes early, and just call a duck a duck and consider an on-time flight to be an on-time flight). Cancelled and diverted flights are not assigned a delay value.
However, when we count delays, we will only include flights delayed by more than 15 minutes, which adheres to the Federal Aviation Administration’s definition of a delay.
II. Data Source
Our data source was the Bureau of Transportation Statistics’ Carrier On-Time Performance archive, which allows customized downloads of flight delay statistics.
Of course, flight data is rigorously maintained and regulated, so the veracity and accuracy of this data isn’t a major concern.
There are a plethora of variables, including different types of delay times, taxi times, flight times, origin, destination, etc. There’s much more than we needed here. We will consider a total of approximately 1.6 million flights across these analyses.
III. Data Transformation
We customized our variables using the BTS’ interface, then downloaded .csv data for four separate Novembers: 2015, 2016, 2017, and 2018. We then manually combined the five-day Thanksgiving holidays into one file, thanksgiving.csv.
We did the same thing for the each of the four 25 day periods ending with the post-Thanksgiving Sunday.
These data cleaning files can be found under getdata.R and getdataALLnov.R in our GitHub repo.
IV. Missing Values
We will examine missing data patterns for 150 random rows of our data (which is about as many as we can view at once).
tidyflights <- thanksgiving[sample(dim(thanksgiving)[1],150),] %>%
mutate(id=as.factor(X.1)) %>%
select(-X.1) %>%
gather(key, value, -id) %>%
mutate(Missing = ifelse(is.na(value), "Yes", "No")) %>%
filter(key != 'X')
ggplot(tidyflights, aes(x = key, y = fct_rev(id), fill = Missing)) +
geom_tile(color = "white") +
ggtitle("Missing Data For Thanksgiving Flights") +
scale_fill_viridis_d() +
theme_bw() +
theme(
axis.text.x = element_blank(),
axis.ticks = element_blank(),
axis.title = element_blank(),
legend.title = element_text(hjust = 0.5),
plot.title = element_text(hjust = 0.5)
) +
coord_flip()
It seems that a very small number of flights are missing a curiously large amount of data, which turn out to be cancelled flights. We won’t be able to include these in our delay time analysis, since these flights have no delay data to speak of. For this reason, we will examine cancelled flights independently later.
The only other missing data is for weather delays and carrier delays - which, judging by the patterns, we will assume were labeled NAs when no specific delay category applied a flight. Outside of that, it seems that NAs shouldn’t be a major barrier to our analysis.
With both of these facts in mind, we will mostly pass over NAs (i.e. not include them in means) when computing our statistics, and keep this structure in mind when we consider delay types later.
V. Results
Without any further ado, we will now present our research. Happy flying!
Which Airline Is Best For Thanksgiving?
Let’s start by examining the simplest decision an air traveler can make: which airline to fly during the holiday.
thanksgiving = thanksgiving %>% mutate(OP_UNIQUE_CARRIER = recode(OP_UNIQUE_CARRIER, "9E" = "Endeavor Air",
"EV" = "ExpressJet",
"G4" = "Allegiant",
"HA" = "Hawaiian",
"MQ" = "Envoy",
"OH" = "PSA",
"OO" = "SkyWest",
"VX" = "Virgin America",
"YV" = "Mesa",
"YX" = "Republic",
"AA" = "American",
"AS" = "Alaska",
"B6" = "JetBlue",
"DL" = "Delta",
"F9" = "Frontier",
"NK" = "Spirit",
"UA" = "United",
"WN" = "Southwest"))
alldates = alldates %>% mutate(OP_UNIQUE_CARRIER = recode(OP_UNIQUE_CARRIER, "9E" = "Endeavor Air",
"EV" = "ExpressJet",
"G4" = "Allegiant",
"HA" = "Hawaiian",
"MQ" = "Envoy",
"OH" = "PSA",
"OO" = "SkyWest",
"VX" = "Virgin America",
"YV" = "Mesa",
"YX" = "Republic",
"AA" = "American",
"AS" = "Alaska",
"B6" = "JetBlue",
"DL" = "Delta",
"F9" = "Frontier",
"NK" = "Spirit",
"UA" = "United",
"WN" = "Southwest"))by_airline = thanksgiving %>% group_by(OP_UNIQUE_CARRIER) %>% summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE)) %>% arrange(-delay)
ggplot(by_airline,aes(x=fct_reorder(OP_UNIQUE_CARRIER,delay),y=delay)) +
geom_col(position='dodge',fill='black') +
theme_bw() +
xlab('') +
ylab('Average Arrival Delay (Minutes)') +
ggtitle('Average Thanksgiving Holiday Flight Delays') +
theme(plot.title = element_text(hjust = 0.5,size=15)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(panel.grid.major.y = element_blank(), panel.grid.minor.y = element_blank()) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(axis.ticks.y = element_blank()) +
coord_flip()
Ultra low-cost carrier Allegiant leads the pack in Thanksgiving delays, which isn’t surprising considering its reputation (but wait, more on this in a second!). Regional airline Mesa Airlines comes next. Hawaiian Airlines has the fewest delays of any airline.
It is clear that focusing on large carriers will make the most sense for our purposes, since consumers are typically unfamiliar with regional airlines that fly under either a single mainline carrier’s brand or multiple brands (when’s the last time you booked a Mesa Airlines flight directly?) and those airlines tend to be available for only a small set of routes (Hawaiian Airlines won’t do you much good unless you’re flying to Hawaii), whereas a larger carrier’s network is likely to cover most major routes.
Among the three major legacy mainline carriers, Delta has the fewest delays, followed by American, and United performs by far the worst. Southwest performs a little better than American, and Alaska performs a little worse than American.
Does this vary by year, though? Examining this is difficult because of the structure of the data, as you’ll see below.
by_year = thanksgiving %>% group_by(OP_UNIQUE_CARRIER,year) %>% summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE))
ggplot(by_year,aes(x=fct_reorder(OP_UNIQUE_CARRIER,delay),y=delay,fill=as.factor(year))) +
geom_col(position=position_dodge2(preserve = "single",width=1)) +
theme_bw() +
xlab('') +
ylab('Average Arrival Delay (Minutes)') +
labs(fill = 'Year') +
coord_flip() +
ggtitle('Average Thanksgiving Holiday Flight Delays') +
scale_fill_colorblind() +
theme(plot.title = element_text(hjust = 0.5,size=23)) +
theme(legend.title = element_blank(),legend.text = element_text(size=11)) +
guides(fill = guide_legend(reverse=T)) +
theme(axis.ticks.y = element_blank()) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=14)) +
theme(panel.grid.major.y = element_blank(), panel.grid.minor = element_blank()) 
We will examine year-to-year patterns (particularly the horrors of Thanksgiving Sunday 2018) later, but for now, this graph provides a curious revelation: Allegiant did not have any flights during the Thanksgiving holiday before 2018. No wonder its on time performance was so terrible - it only flew on Thanksgiving during the year when delays were by far the worst.
This graph is here to demonstrate that, as we expected, focusing on large, national, major mainline airlines that flew for all four years will make the most sense for our purposes.
We will apply the following criteria for removing airlines:
- Must have flown flights during Thanksgiving all four years
- No regional or heavily region-specific airlines (regional airlines would be contracted carriers such as Mesa, heavily region-focused airlines would be like Hawaiian Airlines).
- No inactive airlines
For this reason, we have decided to remove the following airlines:
- Endeavor Air
- ExpressJet
- Allegiant
- Hawaiian Airlines
- Envoy Air
- PSA Airlines
- SkyWest Airlines
- Virgin America
- Mesa Airlines
- Republic Airways
Which leaves us with the following eight airlines:
- American Airlines
- Delta Airlines
- United Airlines
- Southwest Airlines
- Alaska Airlines
- JetBlue
- Spirit Airlines
- Frontier Airlines
These happen to be the eight largest passenger airlines in the United States, so this arrangement makes sense.
We will present these airlines graphically sorted left-to-right and top-to-bottom from the largest airline to the smallest (as they are above, with American the largest and Frontier the smallest), and keep this consistent throughout the presentation. Merely for design purposes and ease of tracking, the coloring of an airline’s bars will reflect a primary color of the airline (which means, of course, no matter what your eyesight, these colors will look the same as they do on the planes!).
thanksgiving = thanksgiving %>% filter(OP_UNIQUE_CARRIER %in% c(
'American',
'Delta',
'United',
'Southwest',
'Alaska',
'JetBlue',
'Spirit',
'Frontier'))
alldates = alldates %>% filter(OP_UNIQUE_CARRIER %in% c(
'American',
'Delta',
'United',
'Southwest',
'Alaska',
'JetBlue',
'Spirit',
'Frontier'))Regionality of Delays
Let’s now compare flights based on the regions of their departure airports (since most bad airline delays are caused at the origin, as once a flight it in the air it’s hard to delay it much more due to fuel concerns).
First, we note that we are still dealing with the delays when the flight arrives at its destination, even though we are comparing origin airports here (this is done to maintain consistency across our analyses).
First, let’s examine how delays vary across different states.
thanksgiving_region <- thanksgiving %>% mutate(region = if_else(DEST_STATE_ABR=="AK","West",
if_else(DEST_STATE_ABR=="AL","South",
if_else(DEST_STATE_ABR=="AR","South",
if_else(DEST_STATE_ABR=="AZ","West",
if_else(DEST_STATE_ABR=="CA","West",
if_else(DEST_STATE_ABR=="CO","West",
if_else(DEST_STATE_ABR=="CT","Northeast",
if_else(DEST_STATE_ABR=="DC","South",
if_else(DEST_STATE_ABR=="DE","South",
if_else(DEST_STATE_ABR=="FL","South",
if_else(DEST_STATE_ABR=="GA","South",
if_else(DEST_STATE_ABR=="HI","West",
if_else(DEST_STATE_ABR=="IA","Midwest",
if_else(DEST_STATE_ABR=="ID","West",
if_else(DEST_STATE_ABR=="IL","Midwest",
if_else(DEST_STATE_ABR=="IN","Midwest",
if_else(DEST_STATE_ABR=="KS","Midwest",
if_else(DEST_STATE_ABR=="KY","South",
if_else(DEST_STATE_ABR=="LA","South",
if_else(DEST_STATE_ABR=="MA","Northeast",
if_else(DEST_STATE_ABR=="MD","South",
if_else(DEST_STATE_ABR=="ME","Northeast",
if_else(DEST_STATE_ABR=="MI","Midwest",
if_else(DEST_STATE_ABR=="MN","Midwest",
if_else(DEST_STATE_ABR=="MO","Midwest",
if_else(DEST_STATE_ABR=="MS","South",
if_else(DEST_STATE_ABR=="MT","West",
if_else(DEST_STATE_ABR=="NC","South",
if_else(DEST_STATE_ABR=="ND","Midwest",
if_else(DEST_STATE_ABR=="NE","Midwest",
if_else(DEST_STATE_ABR=="NH","Northeast",
if_else(DEST_STATE_ABR=="NJ","Northeast",
if_else(DEST_STATE_ABR=="NM","West",
if_else(DEST_STATE_ABR=="NV","West",
if_else(DEST_STATE_ABR=="NY","Northeast",
if_else(DEST_STATE_ABR=="OH","Midwest",
if_else(DEST_STATE_ABR=="OK","South",
if_else(DEST_STATE_ABR=="OR","West",
if_else(DEST_STATE_ABR=="PA","Northeast",
if_else(DEST_STATE_ABR=="RI","Northeast",
if_else(DEST_STATE_ABR=="SC","South",
if_else(DEST_STATE_ABR=="SD","Midwest",
if_else(DEST_STATE_ABR=="TN","South",
if_else(DEST_STATE_ABR=="TX","South",
if_else(DEST_STATE_ABR=="UT","West",
if_else(DEST_STATE_ABR=="VA","South",
if_else(DEST_STATE_ABR=="VT","Northeast",
if_else(DEST_STATE_ABR=="WA","West",""
)))))))))))))))))))))))))))))))))))))))))))))))))
thanksgiving_region$region<- ifelse(thanksgiving_region$DEST_STATE_ABR=="WI" & thanksgiving_region$region =="","Midwest",
ifelse(thanksgiving_region$DEST_STATE_ABR=="WV" & thanksgiving_region$region =="","South",
ifelse(thanksgiving_region$DEST_STATE_ABR=="AL" & thanksgiving_region$region =="","South",
ifelse(thanksgiving_region$DEST_STATE_ABR=="WY" & thanksgiving_region$region=="","West",thanksgiving_region$region))))
thanksgiving_region<-thanksgiving_region %>% filter(!region %in% c(""))
avg_delay<- thanksgiving_region %>% group_by(DEST_STATE_ABR) %>% summarise(avg_delay = mean(ARR_DELAY_NEW,na.rm=T))
statepop2 <- statepop %>% left_join(unique(avg_delay[,c("DEST_STATE_ABR","avg_delay")]),by = c("abbr" = "DEST_STATE_ABR"))
statepop2$avg_delay <- ifelse(statepop2$abbr %in% c("DE","DC"),mean(statepop2$avg_delay,na.rm=T),statepop2$avg_delay)
plot_usmap(data = statepop2, values = "avg_delay") + theme(legend.position = "right") +
scale_fill_continuous(name = "Average Delay (Minutes)") + labs(title = "Average Thanksgiving Holiday Flight Delays") +
theme(plot.title = element_text(hjust = 0.5,face="bold"),
legend.title = element_text(hjust=0.5))
It’s difficult to make a whole lot of this map, other than to say that North Dakota is a particularly bad offender (so maybe don’t plan your Thanksgiving travel through Fargo!).
Let’s try grouping by region. We’ll use a slight tweak of the official U.S. Census Bureau definitions for regions of the United States, which we’ve outlined below.
statepop2 <- statepop %>% left_join(unique(thanksgiving_region[,c("DEST_STATE_ABR","region")]),by = c("abbr" = "DEST_STATE_ABR"))
statepop2$region <- ifelse(statepop2$abbr %in% c("DE","DC"),"South",statepop2$region)
plot_usmap(data = statepop2, values = "region") + theme(legend.position = "right") +
scale_fill_discrete(name = "Regions") + labs(title = "Regions") +
theme(plot.title = element_text(hjust = 0.5,face="bold"),
legend.title = element_text(hjust=0.5))
Let’s take a look at how the delays in these regions compare.
by_region_delay2 = thanksgiving_region %>% group_by(region) %>% summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE))
ggplot(by_region_delay2,aes(x=region,y=delay)) +
geom_col(position=position_dodge2(preserve = "single",width=1),fill='black') +
theme_bw() +
xlab('') +
ylab('Average Arrival Delay (Minutes)') +
ggtitle('Thanksgiving Holiday Flight Delays By Region')+
theme(plot.title = element_text(hjust = 0.5,size=16)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(axis.ticks.x = element_blank())
Delays appear to be the worst in the Northeast and West - which makes sense given how densely populated those areas are (and how many airline hubs there are in those areas).
Next, we will compare the number of flights delayed in different regions across the different years (again, a “delayed flight” is one that arrives more than 15 minutes late).
In this graph, we will examine how each region’s delays were spread across the four years (assuming, for the purposes of this analysis, that there were a fairly equal number of flights all four years for each of the regions). We recognize that this creates a somewhat unusual mosaic plot, but it serves its purpose well.
by_region_delay = thanksgiving_region %>%
mutate(region =
ifelse(region == 'Northeast','NE',
ifelse(region == 'South','S',
ifelse(region == 'West','W',
ifelse(region=='Midwest','MW',NA))))) %>%
filter(ARR_DELAY_NEW>15) %>%
group_by(year,region) %>%
summarise(Freq=n())
vcd::mosaic(year ~ region,
gp = grid::gpar(fill = c('gold','lightblue','darkgreen','black'),
col = "white"),
spacing = vcd::spacing_equal(sp = unit(0, "lines")),
by_region_delay,
labeling=vcd::labeling_border(rot_labels = c(0,0,0,0)),
main = 'Flights Delays: Grouped By Region')
The mysterious 2018 mess had a bigger impact on the Midwest, Northeast, and South than it did on the West. This suggests that it was probably weather-related, but we’ll explore that later.
Outside of this, it seems that delays are somewhat pocketed to different regions, but there are ripple effects that prevent a truly regional analysis.
This could change when we compare airlines regionally, though.
by_flight_delay = thanksgiving_region %>% group_by(OP_UNIQUE_CARRIER,region) %>% summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE))
ggplot(by_flight_delay,aes(x=fct_rev(fct_relevel(OP_UNIQUE_CARRIER,'American',
'Delta',
'United',
'Southwest',
'Alaska',
'JetBlue',
'Spirit',
'Frontier')),y=delay,fill=OP_UNIQUE_CARRIER)) +
geom_col(position=position_dodge2(preserve = "single",width=1)) +
theme(axis.title = element_text(size=12), axis.text.x = element_text(angle = 90, hjust = 1)) +
xlab('') +
coord_flip() +
theme_bw() +
facet_wrap(~region,nrow=4)+
ylab('Average Arrival Delay (Minutes)') +
ggtitle('Thanksgiving Holiday Flight Delays By Region')+
theme(plot.title = element_text(hjust = 0.5,size=16)) +
theme(legend.position = 'none') +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.y = element_blank(), panel.grid.minor.y = element_blank()) +
scale_colour_manual(
values = c('American'='#787890','Alaska'='#8caa48','JetBlue'='#003876',
'Delta'='#C01933','Frontier'='#248168','Spirit'='#feea67',
'United'='#8099d0','Southwest'='#5e2e24'),
aesthetics = c("colour", "fill")) +
theme(axis.ticks.y = element_blank()) +
theme(strip.text = element_text(size=14))
There are a lot of things going on here, but it’s fairly clear from this graph that airline performance does vary by region. For instance, Frontier is much worse than United for flights leaving from the South, but United performs worse when it comes to flights leaving from the Midwest (although this could be due to the 2018 mess we’ll discuss later).
We see that in the South, Delta and Southwest have the best performance, followed by American, which is intuitive as they all have giant hubs in that region.
We’ll try a more granular approach to gain more insight.
Comparing Airports
Let’s now compare delays across airports, again with the caveat that these refer to arrival delays even though we are comparing departure airports.
thanksgiving_region_dot_plot<-thanksgiving_region[thanksgiving_region$year!=2018,] %>% group_by(DEST) %>%
summarise(delay=mean(ARR_DELAY_NEW,na.rm=TRUE)) %>% arrange(-delay)
thanksgiving_region_dot_plot<-thanksgiving_region_dot_plot[c(1:50),]
thanksgiving_region_dot_plot<-unique(merge(thanksgiving_region_dot_plot,thanksgiving_region[,c("DEST","region")],by="DEST",all.x = TRUE))
theme_dotplot <- theme_bw(14) +
theme(axis.text.y = element_text(size = rel(.75)),
axis.ticks.y = element_blank(),
panel.grid.major.x = element_blank(),
panel.grid.major.y = element_line(size = 0.5),
panel.grid.minor.x = element_blank(),
axis.text = element_text(size=11),
axis.title = element_text(size=13))
# create the plot
ggplot(thanksgiving_region_dot_plot, aes(x = delay, y = reorder(DEST, delay),color=region)) +
geom_point(color = "black",size=2) +
theme_dotplot +
xlab("\nAverage Arrival Delay (Minutes)") +
ylab("Departure Airport\n") +
ggtitle("Thanksgiving Holiday Flight Delays By Airport")+
theme(plot.title = element_text(hjust = 0.5)) 
The worst-affected airports are Rapid City, SD (RAP), Santa Barbara, CA (SBA), and Fargo, ND (FAR).
Of course, many of these are small airports, so probably either (a) you’re forced to fly out of them to get where you’re going to (so this information won’t help your decision-making, and will only needlessly scare you), or (b) they’re of no relevance to you because you’re unlikely to find an itinerary that involves changing planes there.
Thus, we will focus our analysis on flights originating from the 10 biggest airports (measured specifically by the number of flights originating from there during the Thanksgiving holiday), which are much more likely to facilitate flight connections, and thus more likely to factor into the general traveler’s plans.
We will also compare them to their pre-Thanksgiving performance, in order to glean more insight into which ones get worse during the holiday and which ones get better.
top_10_airports<-alldates %>% group_by(ORIGIN) %>% summarise(cnt =n()) %>% arrange(-cnt)
top_10_airports<-top_10_airports$ORIGIN[1:10]
alldates_top10<-alldates[alldates$ORIGIN %in% top_10_airports,]
by_airport_delay_tg = alldates_top10 %>% filter(period == "Thanksgiving") %>% group_by(ORIGIN,period) %>% summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE))
by_airport_delay_ntg = alldates_top10 %>% filter(period =="Pre-Thanksgiving") %>% group_by(ORIGIN,period) %>% summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE))
by_airport_delay<-bind_rows(by_airport_delay_tg,by_airport_delay_ntg)
ggplot(by_airport_delay,aes(x=fct_reorder(ORIGIN,delay),y=delay,fill=fct_rev(period))) +
geom_col(position=position_dodge2(preserve = "single",width=1)) +
theme_bw() +
xlab('') +
ylab('Average Arrival Delay (Minutes)') +
ggtitle('Delays For Flights From Busiest 10 US Airports') +
labs(subtitle = '20 Days Before & Five Days During Thanksgiving Holiday') +
theme(plot.title = element_text(hjust = 0.5,size=14)) +
theme(plot.subtitle = element_text(hjust = 0.5,size=12)) +
theme(legend.title = element_blank(),legend.text = element_text(size=11)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
guides(fill = guide_legend(reverse=T)) +
scale_colour_manual(
values = c('red','blue'),
aesthetics = c("colour", "fill")) +
theme(panel.grid.major.y = element_blank(), panel.grid.minor.y = element_blank()) +
theme(axis.ticks.y = element_blank()) +
coord_flip() 
Of the top ten airports, nine get better and have shorter average delays during the five-day Thanksgiving holiday than the 20 days leading up to it. The only one that gets worse is San Francisco.
This is quite a revelation! It raises a lot of questions. Could there be another factor in play here that’s causing this surprising data?
We’ll examine a lot of possibilities throughout the rest of this analysis. First, we’ll compare flight cancellations per day between the “Pre-Thanksgiving” and “Thanksgiving” periods - perhaps these flights are delayed less on average because a lot more are cancelled?
cancellations_by_aiport = alldates_top10 %>%
group_by(ORIGIN,period) %>%
summarize(cancelled= sum(CANCELLED,na.rm=TRUE)) %>%
mutate(divider = ifelse(period=='Thanksgiving',20,80)) %>%
mutate(cancelled_per_day = cancelled/divider)
ggplot(cancellations_by_aiport,aes(x=fct_reorder(ORIGIN,cancelled_per_day),y=cancelled_per_day,fill=fct_rev(period))) +
geom_col(position=position_dodge2(preserve = "single",width=1)) +
theme_bw() +
xlab('') +
ylab('Number of Cancellations Per Day') +
ggtitle('Flight Cancellations Per Day For 10 Busiest US Airports') +
labs(subtitle = '20 Days Before vs. Five Days During Thanksgiving Holiday') +
theme(plot.title = element_text(hjust = 0.5,size=14)) +
theme(plot.subtitle = element_text(hjust = 0.5,size=12)) +
theme(legend.title = element_blank(),legend.text= element_text(size=11)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
guides(fill = guide_legend(reverse=T)) +
scale_colour_manual(
values = c('red','blue'),
aesthetics = c("colour", "fill")) +
theme(axis.ticks.y = element_blank()) +
theme(panel.grid.major.y = element_blank(), panel.grid.minor.y = element_blank()) +
coord_flip() 
It seems that most of these airports either have fewer flight cancellations per day or about the same number cancelled per day - except Chicago, which sees a massive spike as it leapfrogs San Francisco to the top.
Chicago’s oddly high number of cancellations ties into the 2018 anomaly (don’t worry, we’ll address it shortly!).
Now we’ll start tackling a question perhaps even juicier than airline choice - when do you travel?
Comparing Days & Times Of Travel
We analyze how airline delays progress leading up to and following the day of Thanksgiving itself. This will help us ascertain whether there is a general trend of airline delays and cancellations on specific days.
Our data captures multiple potential causes for delays: it is subdivided into the different categories such as weather delays, carrier delays, taxi delays, etc. We’ll get to a couple of these in a moment.
First, though, let’s examine how delays change based on time of day.
thanksgiving2 = thanksgiving %>%
mutate(DAY_OF_WEEK = recode(DAY_OF_WEEK,
"3" = "Wednesday",
"4" = "Thursday",
"5" = "Friday",
"6" = "Saturday",
"7" = "Sunday"))
thanksgiving2$DAY_OF_WEEK = factor(thanksgiving2$DAY_OF_WEEK,
levels = c("Wednesday", "Thursday", "Friday", "Saturday", "Sunday"))
delay_by_dept_time = thanksgiving2 %>%
group_by(DEP_TIME) %>%
summarise(arr_delay = mean(ARR_DELAY_NEW, na.rm=TRUE))
delay_by_dept_time$DEP_TIME = str_pad(delay_by_dept_time$DEP_TIME, 4, pad = "0")
delay_by_dept_time$DEP_TIME = substr(delay_by_dept_time$DEP_TIME, 1, 2)
delay_by_dept_time = delay_by_dept_time %>%
mutate(DEP_TIME = replace(DEP_TIME, DEP_TIME == "24", "00")) %>%
group_by(DEP_TIME) %>%
summarise(arr_delay_mean = mean(arr_delay))
delay_by_dept_time = delay_by_dept_time %>% filter(!is.na(DEP_TIME))
ggplot(delay_by_dept_time, aes(x = DEP_TIME, y = arr_delay_mean)) +
geom_bar(stat = "identity",fill='black') +
coord_polar(theta = "x") +
ggtitle("Average Arrival Delay Based On Departure Time") +
labs(x = "Departure Hour (24 Hour Format)",
y = "Average Arrival Delay (Minutes)") +
theme_bw() +
theme(plot.title = element_text(hjust = 0.5)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(legend.title = element_text(hjust = 0.5))
After plotting the departure hour of flights on a radial axis, we get a surprising result. Contrary to our expectation, the average delay of flights is much higher for flights leaving overnight, between 1AM-4AM, drastically drops starting 6AM, and stays relatively steady until nighttime. It seems like we can almost universally state that the later you leave, the worse - and you may want to put some extra effort into avoiding red-eyes.
Next, we’ll examine how cancellations have fluctuated across different days across different years.
cancellations_by_day = thanksgiving2 %>%
group_by(year, DAY_OF_WEEK) %>%
summarise(cancellations = sum(CANCELLED, na.rm=TRUE))
ggplot(cancellations_by_day, aes(x = fct_rev(DAY_OF_WEEK), y = cancellations)) +
geom_bar(stat = "identity",fill='black') +
facet_wrap(~year,nrow=4) +
ggtitle("Total Flight Cancellations During Thanksgiving Week (2015-2018)") +
labs(x = "",
y = "Total Cancellations") +
theme_bw() +
theme(plot.title = element_text(hjust = 0.5)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(legend.title = element_text(hjust = 0.5)) +
theme(panel.grid.major.y = element_blank(), panel.grid.minor.y = element_blank()) +
theme(axis.ticks.y = element_blank()) +
coord_flip()
Ah, there’s our culprit!
We’ve found our outlier: Thanksgiving Sunday 2018. Clearly, something must have caused all of these cancellations, and may have skewed our data.
Let’s see what was happening on Thanksgiving Sunday, 2018.
The Thanksgiving Sunday 2018 Flight Disaster
sunday_2018 = thanksgiving2 %>%
filter(DAY_OF_WEEK == 'Sunday' & year == 2018) %>%
drop_na(ARR_DELAY_NEW, WEATHER_DELAY, CARRIER_DELAY) %>%
group_by(OP_UNIQUE_CARRIER) %>%
summarise(arr_delay = mean(ARR_DELAY_NEW, na.rm=TRUE),
weather_delay = mean(WEATHER_DELAY, na.rm = TRUE),
carrier_delay = mean(CARRIER_DELAY, na.rm = TRUE)) %>%
mutate(arr_delay = arr_delay - (weather_delay + carrier_delay)) %>%
rename("Arrival Delay" = arr_delay,
"Weather Delay" = weather_delay,
"Carrier Delay" = carrier_delay)
sunday_2018 = gather(sunday_2018, "delay_type", "delay_time", 2:4)
ggplot(sunday_2018, aes(x = reorder(OP_UNIQUE_CARRIER, -delay_time),
fill = delay_type, y = delay_time)) +
geom_bar(position = "stack", stat = "identity") +
ggtitle("Cause of Flight Delays, Thanksgiving Sunday 2018") +
labs(x = "",
y = "Average Arrival Delay (Minutes)") +
theme_bw() +
scale_fill_colorblind() +
theme(plot.title = element_text(hjust = 0.5)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
guides(fill = guide_legend(title="Delay Type")) +
theme(legend.title = element_text(hjust = 0.5)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(axis.ticks.x = element_blank())
United Airlines suffered major weather delays that day. Every other type of delay cause stayed fairly consistent across most airlines, but United Airlines clearly struggled the most.
During our deep dive into this issue, we found a news article online talking specifically about the day and the year we were observing.
A snippet from this article:
Most of those (delays) came in the Midwest, where a winter storm was bringing snow, ice and rain to a swath of the Great Plains and Midwest. Blizzard conditions were possible Sunday across parts of Iowa, Illinois, Missouri and Wisconsin.
O’Hare is a major connecting hub for both United and American, each of which had canceled hundreds of mainline and regional flights. By FlightAware’s count. the cancellations accounted for more than 25 percent of Sunday’s entire schedule at O’Hare. For Monday, more than 180 flights – about 5 percent of the day’s schedule – had already been grounded by Sunday evening.
Looking at airports may give us more insight.
sunday_airports = alldates_top10 %>%
filter(period == 'Thanksgiving',DAY_OF_WEEK==7,year==2018) %>%
drop_na(ARR_DELAY_NEW, WEATHER_DELAY, CARRIER_DELAY) %>%
group_by(ORIGIN) %>%
summarise(arr_delay = mean(ARR_DELAY_NEW, na.rm=TRUE),
weather_delay = mean(WEATHER_DELAY, na.rm = TRUE),
carrier_delay = mean(CARRIER_DELAY, na.rm = TRUE)) %>%
mutate(arr_delay = arr_delay - (weather_delay + carrier_delay)) %>%
rename("Arrival Delay" = arr_delay,
"Weather Delay" = weather_delay,
"Carrier Delay" = carrier_delay)
sunday_2018_2 = gather(sunday_airports, "delay_type", "delay_time", 2:4)
ggplot(sunday_2018_2, aes(x = reorder(ORIGIN, -delay_time),
fill = delay_type, y = delay_time)) +
geom_bar(position = "stack", stat = "identity") +
ggtitle("Cause of Flight Delays, Thanksgiving Sunday 2018") +
labs(x = "Airport",
y = "Average Arrival Delay (Minutes)") +
theme_bw() +
xlab('') +
scale_fill_colorblind() +
theme(plot.title = element_text(hjust = 0.5)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
guides(fill = guide_legend(title="Delay Type")) +
theme(legend.title = element_text(hjust = 0.5)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(axis.ticks.x = element_blank()) 
Chicago O’Hare, San Francisco, and Denver are all United hubs and had significant weather delays, so it makes sense that United would have suffered the most from the weather.
This backs our hypothesis that the delay surge on this particular day was a product of weather, not a product of the Thanksgiving rush.
This is an important caveat to our data. We opted to keep this data in our dataset, as weather is just one of many challenges airlines face - but it certainly skews the data against United a bit.
Perhaps we should examine how badly 2018 penalized United (and maybe other airlines).
not_2018 = thanksgiving %>%
mutate(period = ifelse(year<2018,'2015-2017','2018')) %>%
group_by(period,OP_UNIQUE_CARRIER) %>%
summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE)) %>%
arrange(-delay)
ggplot(not_2018,aes(x=OP_UNIQUE_CARRIER,y=delay,fill=period)) +
geom_col(position='dodge') +
theme_bw() +
xlab('') +
ylab('Average Arrival Delay (Minutes)') +
ggtitle('Thanksgiving Holiday Flight Delays: 2015-17 vs. 2018') +
scale_colour_manual(
values = c('red','blue'),
aesthetics = c("colour", "fill")) +
theme(plot.title = element_text(hjust = 0.5,size=15)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(axis.ticks.y = element_blank()) +
theme(legend.position = 'bottom',legend.title=element_blank())
United would still be among the worst airlines for Thanksgiving OTP even without 2018, but 2018 (particularly that Sunday) made United look worse than it normally is. A similar statement could be made about Frontier and JetBlue.
Next, we’ll move to comparing the Thanksgiving and ‘Pre-Thanksgiving’ holiday periods.
The Major Revelation: Maybe Thanksgiving Isn’t So Bad
We will now answer a vital question: how has airline OTP during the Thanksgiving holiday compared to airline OTP outside of the Thanksgiving holiday?
We will research this question by comparing the 20 days before the Thanksgiving travel rush to the five days of the Thanksgiving travel period, in order ensure that we stay in a season with similar weather and travel patterns. We note, however, that likely captured some early Thanksgiving travel (which could be equally or perhaps even moreso plagued by delays). We’ll discuss this more in a bit.
We also share a caveat (which applies to all this research) that the four years since the recent major US airline mergers completed constitute a small sample size, and major weather events and airline disruptions could (and as we saw, probably did) cause some skewness in this data.
That being said, let’s compare Thanksgiving and pre-Thanksgiving.
by_period_all = alldates %>% group_by(period) %>% summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE))
ggplot(by_period_all,aes(x=period,y=delay,fill=period)) +
geom_col(position='dodge') +
theme_bw() +
xlab('') +
ylab('Average Arrival Delay (Minutes)') +
ggtitle('Major Eight Airline Delays') +
labs(subtitle = '20 Days Before Thanksgiving Holiday vs. Five Days Of Thanksgiving Holiday') +
scale_colour_manual(
values = c('red','blue'),
aesthetics = c("colour", "fill")) +
theme(plot.title = element_text(hjust = 0.5)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(legend.position = "none") +
theme(axis.ticks.x = element_blank())
A stunner: over the past four years, airlines have actually gotten better at on-time performance during Thanksgiving.
There’s an important caveat worth mentioning here: the data we are using considers any early flight to be simply “on-time” (and does not give it a negative delay value), as we believe that it makes the most sense to call a spade a spade and consider any flight that isn’t late to be on time.
However, flights are typically early when all goes right (due to schedule padding), so it’s possible that Thanksgiving causes a lot of shorter “delays” that result in flights being on-time instead of early, meaning Thanksgiving flights aren’t really quicker.
Those are fairly flimsy caveats, though, so our result stands: at the very least, it seems evident that delays don’t get worse during Thanksgiving (at least for flights that aren’t cancelled or diverted, which we’ll address later).
Now let’s examine how this trend plays out for different airlines.
by_airline_n = alldates %>% group_by(OP_UNIQUE_CARRIER,period) %>% summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE))
ggplot(by_airline_n,aes(x=fct_relevel(OP_UNIQUE_CARRIER,'American',
'Delta',
'United',
'Southwest',
'Alaska',
'JetBlue',
'Spirit',
'Frontier'),y=delay,fill=period)) +
geom_col(position='dodge') +
theme_bw() +
xlab('') +
ylab('Average Delay') +
ggtitle('Airline Delays In November... Before & During Thanksgiving') +
labs(subtitle = '20 Days Before Thanksgiving Holiday vs. Five Days Of Thanksgiving Holiday') +
scale_colour_manual(
values = c('red','blue'),
aesthetics = c("colour", "fill")) +
theme(plot.title = element_text(hjust = 0.5,size=14)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
guides(fill=guide_legend(title="Period")) +
theme(legend.title = element_blank(),legend.position = 'bottom', legend.direction = 'horizontal',legend.text= element_text(size=11)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(axis.ticks.x = element_blank()) 
delay_c = alldates %>%
group_by(OP_UNIQUE_CARRIER,period) %>%
summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE)) %>%
ungroup() %>%
mutate(delay_change = 100*(delay - lag(delay))/lag(delay)) %>%
filter(period=='Thanksgiving')
ggplot(delay_c,aes(x=fct_relevel(OP_UNIQUE_CARRIER,'American',
'Delta',
'United',
'Southwest',
'Alaska',
'JetBlue',
'Spirit',
'Frontier'),y=delay_change,fill=OP_UNIQUE_CARRIER)) +
geom_col(position='dodge') +
theme_bw() +
xlab('') +
ylab('Percent Change In Average Delay') +
ggtitle('Airline Delay Swings During Thanksgiving') +
labs(subtitle = 'From 20 Days Before Thanksgiving Holiday To Five Days Of Thanksgiving Holiday') +
scale_colour_manual(
values = c('American'='#787890','Alaska'='#8caa48','JetBlue'='#003876',
'Delta'='#C01933','Frontier'='#248168','Spirit'='#feea67',
'United'='#8099d0','Southwest'='#5e2e24'),
aesthetics = c("colour", "fill")) +
theme(plot.title = element_text(hjust = 0.5,size=16)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(legend.position='none') +
theme(axis.ticks.x = element_blank())
All eight airlines see their on-time performance improve during Thanksgiving. Wow!
Outside of that, the most evident (and strangest) trend here is that Spirit Airlines and Frontier Airlines - two ultra low-cost carriers with poor service reputations - have had significantly shorter average flight delays during the Thanksgiving holiday than they havehad prior to the Thanksgiving holiday, particularly Spirit Airlines.
This suggests that there is no additional need to avoid Spirit and Frontier during the Thanksgiving holiday. In fact, one could argue if there’s any time to fly these airlines, it’s Thanksgiving: their OTP has lagged much less during this peak travel period. United (the third-worst airline for OTP) also sees an improvement in performance around Thanksgiving.
Perhaps Thanksgiving has a normalizing effect on these airlines. Frontier technically still has the worst Thanksgiving travel delays of any of these airlines, but they close the gap, and Spirit improves by a remarkable margin.
Let’s not bury the lede here: flight delays are shorter during Thanksgiving. Surprise!
Are there more long delays, though - say, those over 30 minutes? Let’s consider that now.
delays_over_30 = alldates %>% filter(ARR_DELAY_NEW > 30)
by_airline_30 = delays_over_30 %>% group_by(OP_UNIQUE_CARRIER,period) %>% summarize(delay = mean(ARR_DELAY_NEW,na.rm=TRUE))
ggplot(by_airline_30,aes(x=fct_relevel(OP_UNIQUE_CARRIER,'American',
'Delta',
'United',
'Southwest',
'Alaska',
'JetBlue',
'Spirit',
'Frontier'),y=delay,fill=period)) +
geom_col(position='dodge') +
theme_bw() +
xlab('') +
ylab('Average Arrival Delay (Minutes)') +
ggtitle('Delays of Over 30 Minutes') +
labs(subtitle = '20 Days Before Thanksgiving Holiday vs. Five Days Of Thanksgiving Holiday') +
scale_colour_manual(
values = c('red','blue'),
aesthetics = c("colour", "fill")) +
theme(plot.title = element_text(hjust = 0.5,size=18)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
guides(fill=guide_legend(title="Period")) +
theme(legend.title = element_blank(),legend.position = 'bottom', legend.direction = 'horizontal',legend.text = element_text(size=11)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(axis.ticks.x = element_blank())
It seems that long delays are fairly evenly distributed across all airlines and dates, so not much to add there.
What about really long delays though? Which airlines have the most of those, and during which period do these happen the most?
ggplot(alldates) +
geom_boxplot(aes(x = fct_rev(fct_relevel(OP_UNIQUE_CARRIER,'American',
'Delta',
'United',
'Southwest',
'Alaska',
'JetBlue',
'Spirit',
'Frontier')), y = ARR_DELAY_NEW),alpha=0.2) +
coord_flip() +
theme_bw() +
xlab('') +
ylab('Arrival Delay (Minutes)') +
ggtitle('Long Airline Delays During November') +
labs(subtitle = '20 Days Before Thanksgiving Holiday vs. Five Days Of Thanksgiving Holiday') +
scale_colour_manual(
values = c('American'='#787890','Alaska'='#8caa48','JetBlue'='#003876',
'Delta'='#C01933','Frontier'='#248168','Spirit'='#feea67',
'United'='#8099d0','Southwest'='#5e2e24'),
aesthetics = c("colour", "fill")) +
theme(plot.title = element_text(hjust = 0.5,size=18)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
guides(fill=guide_legend(title="Airline")) +
theme(legend.title = element_text(hjust = 0.5)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.y = element_blank(), panel.grid.minor.y = element_blank()) +
theme(axis.ticks.y = element_blank()) +
facet_wrap(~period,nrow=2)
ggplot(alldates) +
geom_boxplot(aes(x = fct_rev(period),y = ARR_DELAY_NEW,fill=period),alpha=0.2) +
theme_bw() +
xlab('') +
ylab('Arrival Delay (Minutes)') +
ggtitle('Long Airline Delays During November') +
labs(subtitle = '20 Days Before Thanksgiving Holiday vs. Five Days Of Thanksgiving Holiday') +
scale_colour_manual(
values = c('red','blue'),
aesthetics = c("colour", "fill")) +
theme(plot.title = element_text(hjust = 0.5,size=16)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.y = element_blank(), panel.grid.minor.y = element_blank()) +
theme(legend.position = "none") +
theme(axis.ticks.y = element_blank()) +
coord_flip()
It’s worth noting that the more flights, the more dots - and there are about four times as many flights during the “Pre-Thanksgiving” period, and the larger airlines have more flights than the smaller airlines in both periods. So, we’re looking for where the dots cluster, rather than the raw number of dots in a given area.
Delta appears to improve in this area during the holiday. American and United remain the worst offenders. Interestingly, Spirit again sees quite an improvement during the holiday period, with far fewer serious delays. There isn’t too much change overall in the number of serious delays between the two periods.
But now… what about the flights that aren’t delayed, but just get flat-out cancelled, and thus don’t even don’t have a delay value to factor in? Maybe a lot of flights get cancelled during Thanksgiving, and that removes bad delays from the sample space?
Let’s take a look.
cancellations = alldates %>%
group_by(period) %>%
summarize(cancelled= sum(CANCELLED,na.rm=TRUE)) %>%
mutate(divider = ifelse(period=='Thanksgiving',20,80)) %>%
mutate(cancelled_per_day = cancelled/divider)
ggplot(cancellations,aes(x=period,y=cancelled_per_day,fill=period)) +
geom_col(position='dodge') +
theme_bw() +
xlab('') +
ylab('Flights Cancelled Per Day') +
ggtitle('Major Eight Airline Cancellations Per Day In November') +
labs(subtitle = '20 Days Before Thanksgiving Holiday vs. Five Days Of Thanksgiving Holiday') +
scale_colour_manual(
values = c('red','blue'),
aesthetics = c("colour", "fill")) +
theme(legend.position = 'none') +
theme(plot.title = element_text(hjust = 0.5,size=15)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(axis.ticks.x = element_blank()) 
We expected a rub here, but alas, cancellations drop as well during the Thanksgiving holiday. This further confirms our surprising result: at least over the last four years, traveling during Thanksgiving has been less delay- and cancellation-ridden than traveling before Thanksgiving.
Next, we take a look at which airlines stepped up their getting-off-the-ground game the most during Thanksgiving.
cancellations = alldates %>% group_by(OP_UNIQUE_CARRIER,period) %>% summarize(cancelled= sum(CANCELLED,na.rm=TRUE)) %>%
mutate(divider = ifelse(period=='Thanksgiving',20,80)) %>%
mutate(cancelled_per_day = cancelled/divider)
ggplot(cancellations,aes(x=fct_relevel(OP_UNIQUE_CARRIER,'American',
'Delta',
'United',
'Southwest',
'Alaska',
'JetBlue',
'Spirit',
'Frontier'),y=cancelled_per_day,fill=period)) +
geom_col(position='dodge') +
theme_bw() +
xlab('') +
ylab('Cancellations') +
ggtitle('Airline Cancellations Per Day In November') +
labs(subtitle = '20 Days Before Thanksgiving Holiday vs. Five Days Of Thanksgiving Holiday') +
scale_colour_manual(
values = c('red','blue'),
aesthetics = c("colour", "fill")) +
theme(plot.title = element_text(hjust = 0.5,size=16)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(axis.title = element_text(size=12), axis.text = element_text(size=11)) +
theme(legend.title = element_blank(),legend.position='bottom',legend.direction = 'horizontal') +
theme(legend.text=element_text(size=12)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(axis.ticks.x = element_blank())
cancellations_c = alldates %>%
group_by(OP_UNIQUE_CARRIER,period) %>%
summarize(cancellations = sum(CANCELLED,na.rm=TRUE))%>%
mutate(divider = ifelse(period=='Thanksgiving',20,80)) %>%
mutate(cancelled_per_day = cancellations/divider) %>%
ungroup() %>%
mutate(cancellation_change =
100*(cancelled_per_day - lag(cancelled_per_day)) /
lag(cancelled_per_day)) %>%
filter(period=='Thanksgiving')
ggplot(cancellations_c,aes(x=fct_relevel(OP_UNIQUE_CARRIER,'American',
'Delta',
'United',
'Southwest',
'Alaska',
'JetBlue',
'Spirit',
'Frontier'),y=cancellation_change,fill=OP_UNIQUE_CARRIER)) +
geom_col(position='dodge') +
theme_bw() +
xlab('') +
ylab('Percent Change In Number Of Cancellations Per Day') +
ggtitle('Airline Cancellation Swings During Thanksgiving') +
labs(subtitle='Pre-Thanksgiving To Thanksgiving Cancellations Per Day Change') +
scale_colour_manual(
values = c('American'='#787890','Alaska'='#8caa48','JetBlue'='#003876',
'Delta'='#C01933','Frontier'='#248168','Spirit'='#feea67',
'United'='#8099d0','Southwest'='#5e2e24'),
aesthetics = c("colour", "fill")) +
theme(plot.title = element_text(hjust = 0.5,size=16)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(legend.position='none') +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank()) +
theme(axis.ticks.x = element_blank())
These differences appear to be nothing more than random variation, and we’ll explain why.
The swing for JetBlue is massive - it went from 18.4 flights cancelled per day before the Thanksgiving holiday to 2.0 flights cancelled per day during the holiday. Such a wild swing doesn’t make logical sense. Unless JetBlue invested massive resources in avoiding delays during this period (which is certainly possible), it’s difficult to imagine what could have caused such a reversal.
This suggests that cancellations are high-variation, “luck of the draw” (or more aptly put, “lack of luck of the draw”) events, and measuring their swings is probably a fool’s errand.
That being said, JetBlue has done an exceptionally good job of getting their flights in the air over the last four Thanksgiving holidays.
It’s also worth noting that Southwest seems to have had extreme cancellation problems during the last four Novembers (perhaps they had one particuarly horrible day), but clearly performed better during the holiday than before the holiday period.
Most airlines appeared to get a bit better at getting their flights in the air, except Delta, which was the only airline out of the eight to have more cancellations during the Thanksgiving holiday. Well done, airlines!
So overall, it certainly seems that travelling before Thanksgiving yields better results than traveling during Thanksgiving, even keeping in mind the caveats that we mentioned earlier.
So let’s see: if we examine the five days of the Thanksgiving travel rush and the 20 days before, do we see a peak in delays at any particular point? Could it be that the worst delays happen 3-5 days before Thanksgiving (i.e. the weekend before plus Monday and Tuesday), rather than during the traditional travel rush?
Let’s see what the graphs say. We’ll examine the 20 ‘Pre-Thanksgiving’ days leading up to the Thanksgiving holiday and the five days of the Thanksgiving holiday together as a single 25 day period.
bydates = alldates %>%
mutate(nday = mday(FL_DATE))
bydates$nday =
ifelse(bydates$year == 2015, bydates$nday - 4,
ifelse(bydates$year == 2016, bydates$nday - 2,
ifelse(bydates$year == 2017, bydates$nday - 1,bydates$nday)))
bydatesgrouped = bydates %>%
group_by(nday,year) %>%
summarize(avgdelay = mean(ARR_DELAY_NEW,na.rm=TRUE))
ggplot(bydatesgrouped,aes(x=nday,y=avgdelay,color=as.factor(year)),) +
geom_line(width=2) +
geom_point() +
geom_vline(xintercept=20.5) +
annotate("text", label = "Thanksgiving Holiday", x = 23.4, y = 22, size = 3, colour = "black") +
theme_bw() +
xlab('Number Day In 25 Day Period Ending With Sunday After Thanksgiving') +
ylab('Average Arrival Delay (Minutes)') +
ggtitle('How Average Major Airline Delays Evolve During November') +
labs(subtitle='20 Days Before Thanksgiving Travel Period + Five Days During Thanksgiving Holiday') +
scale_colour_colorblind() +
theme(plot.title = element_text(hjust = 0.5)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
guides(color=guide_legend(title="Year")) +
theme(legend.title = element_text(hjust = 0.5)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank())
This is odd - we’re seeing a number of peaks before the Thanksgiving holiday even begins. Perhaps we should combine these years and see what trend develops.
bydatesgrouped2 = bydates %>%
group_by(nday) %>%
summarize(avgdelay = mean(ARR_DELAY_NEW,na.rm=TRUE))
ggplot(bydatesgrouped2,aes(x=nday,y=avgdelay)) +
geom_line(width=2) +
geom_point() +
geom_vline(xintercept=20.5) +
annotate("text", label = "Thanksgiving Holiday", x = 23.4, y = 22, size = 3, colour = "black") +
theme_bw() +
xlab('Number Day In 25 Day Period Ending With Sunday After Thanksgiving') +
ylab('Average Arrival Delay (Minutes)') +
ggtitle('How Average Major Airline Delays Evolve During November') +
labs(subtitle='20 Days Before Thanksgiving Travel Period + Five Days During Thanksgiving Holiday') +
scale_colour_colorblind() +
theme(plot.title = element_text(hjust = 0.5)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank())
How strange: while there’s a peak at the end of the Thanksgiving holiday (the final day is that dreaded Thanksgiving Sunday), there’s another peak about a week before Thanksgiving that’s worse than that final Sunday, and much worse than the traditionally-dreaded Thanksgiving Wednesday. Perhaps people are leaving earlier than we thought.
What about cancellations? Let’s take a look.
bydatesgrouped3 = bydates %>%
group_by(nday,year) %>%
summarize(cancellations = sum(CANCELLED,na.rm=TRUE)) %>%
mutate(divider = ifelse(nday>20,5,20)) %>%
mutate(cancelled_per_day = cancellations/divider)
g = ggplot(bydatesgrouped3,aes(x=nday,y=cancelled_per_day,color=as.factor(year)),) +
geom_line(width=2) +
geom_point() +
geom_vline(xintercept=20.5) +
annotate("text", label = "Thanksgiving Holiday", x = 23.4, y = -8, size = 3, colour = "black") +
theme_bw() +
xlab('Number Day In 25 Day Period Ending With Sunday After Thanksgiving') +
ylab('Cancellations Per Day') +
ggtitle('How Cancellations Evolve During November, 2015-2018') +
labs(subtitle='20 Days Before Thanksgiving Travel Period + Five Days Of Thanksgiving Holiday') +
scale_colour_colorblind() +
theme(plot.title = element_text(hjust = 0.5)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
guides(color=guide_legend(title="Year")) +
theme(legend.title = element_text(hjust = 0.5)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank())
g
That horrible 2018 Sunday is making it hard to read this graph, so let’s zoom in a bit.
g + coord_cartesian(ylim=c(0,20))
These peaks are hard to read - let’s try combining these years again.
bydatesgrouped4 = bydates %>%
group_by(nday) %>%
summarize(cancellations = sum(CANCELLED,na.rm=TRUE)) %>%
mutate(divider = ifelse(nday>20,20,80)) %>%
mutate(cancelled_per_day = cancellations/divider)
ggplot(bydatesgrouped4,aes(x=nday,y=cancelled_per_day),) +
geom_line(width=2) +
geom_point() +
geom_vline(xintercept=20.5) +
annotate("text", label = "Thanksgiving Holiday", x = 23.4, y = -8, size = 3, colour = "black") +
theme_bw() +
xlab('Number Day In 25 Day Period Ending With Sunday After Thanksgiving') +
ylab('Cancellations Per Day') +
ggtitle('How Cancellations Evolve During November, 2015-2018') +
labs(subtitle='20 Days Before Thanksgiving Travel Period + Five Days Of Thanksgiving Holiday') +
scale_colour_manual(
values = c('red','blue','darkgreen','orange'),
aesthetics = c("colour", "fill")) +
theme(plot.title = element_text(hjust = 0.5)) +
theme(plot.subtitle = element_text(hjust = 0.5)) +
theme(panel.grid.major.x = element_blank(), panel.grid.minor.x = element_blank())
Again, it doesn’t seem like there’s a massive jump caused by Thanksgiving holiday travel.
These graphs yield an unexpected result - it seems that, generally, the Wednesday before Thanksgiving (day 21) isn’t nearly as bad for delays as the week leading up to Thanksgiving, which appears to be more delay-ridden. The weekend after Thanksgiving is messy too, but is clearly affected by that 2018 mess. 2015-2017 saw a jump in delays on Thanksgiving Sunday, but not really for Wednesday - which many anecdotally consider to be the worst travel day of the year. They could be wrong, at least when it comes to flight delays.
This suggests that leaving a few days early for the Thanksgiving holiday doesn’t make you any less prone to delays, while coming home a day or two before Sunday could make your flight more reliable.
In conclusion, the days around Thanksgiving are relatively calm - it’s the period around the weekend before Thanksgiving and the Sunday after Thanksgiving that cause the most problems for travelers.
VI. Interactive Component
To drive home our analysis, we’ll add two interactive graphs here.
Delays For Various Major City Pairs
First, we offer a graph that will allow you to compare average delays for the airlines flying between any pair of cities that had a direct flight to speak of (if no direct flights occurred between the cities, no data will be shown).
For this graph, we included all airlines to be thorough - not just the eight key airlines we identified earlier. The same caveats mentioned earlier still apply to the remaining airlines we originally removed from the data set.
Delays For Flights Departing From Major Airports
Second, check out our interactive map that will allow you to visualize the average delays for flights out of select major airports across the country.
These visualizations are too complex and customizable to distill into sweeping statements, so we will allow you to draw your own conclusions from them.
VII. Conclusion
There are a plethora of takeaways that have been scattered throughout this analysis, and we won’t needlessly recount every one of them there. Rather, we’ll present this as a succinct summary you can take home and share with your loved ones to better your next Thanksgiving flight plan.
When planning Thanksgiving holiday air travel, don’t obsess about straying from the traditional travel periods - it doesn’t seem that the Wednesday before Thanksgiving is much worse for delays than any other day, and the Sunday after isn’t (usually) all that bad. Don’t bother leaving a few days early to save yourself a travel headache - in fact, you may make it worse by doing that. If you want to beat the rush, you absolutely want to leave earlier in the day - not earlier in the week.
Don’t bother avoiding ultra low-cost carriers like Spirit and Frontier - their performance actually improves during the holiday (and significantly so!), contrary to what you might think. That’s the most noticeable difference between airline OTP before versus during the holiday period.
Your choice of airport shouldn’t matter too much - unless, of course, there’s a storm.
The bottom line is this: don’t fret about airline disasters during the Thanksgiving holiday, because in the years since the last major airline mergers went down in the United States, Thanksgiving holiday travel has actually been less wrought with delays than pre-Thanksgiving travel. Even with a disastrous weather delay factored in (we mentioned this caveat earlier), Thanksgiving flight delays haven’t been as bad as pre-Thanksgiving delays over the last few years. Of course, this analysis was done using a small sample of years that could be affected by other external factors we did not detail here, but the surprising trend we found is difficult to write off.
So, at the very least, we can pretty confidently say Thanksgiving travel isn’t more delay-prone than non-Thanksgiving travel. We hope that the next time you go to see your family and friends for the holiday, you can fly peacefully knowing you’re just as likely to get there on time as you would be normally.