The mail has been slowing down all across the country, but are rural areas suffering worse on-time delivery than urban areas?

That’s the question behind a recent GAO report about the Postal Service entitled “Actions Needed to Make Delivery Performance Information More Complete, Useful, and Transparent.”

The GAO report was done at the request of several senators representing rural areas who have been hearing from their constituents that the mail has been taking much longer to arrive than is normal.

Just today there’s an op-ed in the Idaho State Journal about how the mail delays in Idaho are “unacceptable,” as well as a news report in which Senator Jon Tester of Montana is quoted saying, “Montanans tell me that there are serious delays in mail delivery.”

The GAO criticizes the Postal Service and the Postal Regulatory Commission for not doing enough to ensure that rural areas are getting the same delivery service as urban areas.

Of course, slower delivery times are the norm these days, and the problems are national, not just rural.  The Postal Service relaxed its delivery standards in July 2012 and again in January 2015 as part of its effort to consolidate mail processing plants, and the quarterly service performance reports indicate that the Postal Service is having trouble meeting even these more relaxed standards.

But these lower standards and the poor performance results are not the main focus of the GAO report.  Instead, as its title indicates, the GAO report is about making the service performance reports more “complete, useful, and transparent.”

 

Issues with measurement and reporting

When the GAO went to look at the issues with declining service in rural areas, it found basic problems with the way the Postal Service measures and reports its performance, problems that made it impossible to determine what was going on in rural America.  And since the PRC is supposed to be overseeing service performance, it too came in for its share of criticism from the GAO.

According to the GAO, the USPS performance reports do not include “sufficient analysis to hold USPS accountable for meeting its statutory mission to provide service in all areas of the nation.”  That’s primarily because the reports don’t specifically include information about service performance in rural areas.

The Postal Service told the GAO that it would be “cost prohibitive” to improve reporting on delivery performance in the ways outlined in the report, particularly with respect to differentiating between urban and rural locations.

One of the GAO’s recommendations, therefore, is that Congress should direct the Postal Service to provide estimates for what it would actually cost to change its measurement systems.

In response to the GAO report (even before it was published), the Postal Service and the PRC began working on how they might change things to improve reporting and oversight.  The situation is further complicated by the fact that the Postal Service has proposed significant changes to the way it gathers the data for the performance reports.  So it may be a while before they come up with a system that will measure urban and rural delivery performance — if that is even going to happen at all.

In the meantime, it’s worth looking at the basic question underlying the GAO report: Are rural areas getting worse on-time delivery service than urban areas?

The short answer is perhaps, but not necessarily.

 

The performance reports at first glance

If you look at the service performance reports that the Postal Service shares with the PRC, it appears that districts likely to be very rural often have better performance than highly urbanized districts.

For example, consider the performance report for Single-Piece First Class Mail for the third quarter of 2015, year to date, which can be seen here.  (All of the performance reports are on the PRC website here, now easier to find in response to another GAO criticism.)

For the New York district, which is obviously very urban, 86.2 percent of 2-day mail arrived on time, and 66.3 percent of 3-5 day mail arrived on time.  For the Dakotas district, which is presumably very rural, 95 percent of 2-day mail arrived on time, and 73.8 percent of 3-5 day mail arrived on time. In this case, the rural district had better on-time performance.

Looking deeper into the service performance reports, it’s hard to generalize because it’s not obvious which of the 67 USPS districts are predominantly “rural” and which are mostly “urban.”  That points to a basic issue with analyzing service performance.

As the Postal Service told the GAO, every district contains at least one area that’s urban, so it is problematic to classify any district as either “urban” or “rural.”

But perhaps that’s not necessary.  It might instead be helpful to consider which districts are more rural than others, i.e., which ones have a greater portion of the population living in rural areas.  The U.S. Census can help with figuring that out.

 

Using the Census

The Census website provides data showing the total population for each Zip Code Tabulation Area (ZCTA) in the country.  (A ZCTA is the Census’s geographic representation of a USPS zip code.  A more detailed description is here.)

The Census shows the number of people in each ZCTA living in urban and rural areas.  The Census defines these terms as follows.  (There’s more about the definitions here.)

• An “urban area” is “a densely settled core” that meets minimum density requirements and that has at least 2,500 people (at least 1,500 of which reside outside institutional group quarters).
• There are two types of urban areas: Urbanized Areas (UAs) of 50,000 or more people, and Urban Clusters (UCs) of between 2,500 and 50,000 people.
• “Rural” encompasses all population, housing, and territory not included within an urban area.

These, or course, are not the only definitions around.  As the GAO notes, the Office of Management and Budget generally defines a rural location as a county or equivalent that does not have a core urban area with a population over 10,000.  But let’s stick with the Census, since they have all the numbers.

The Census shows how many people live in urban areas (both Urbanized Areas and Urban Clusters) and how many live in rural areas, and it does so on a zip-by-zip basi

To learn which USPS districts were more rural than others, we downloaded the 2010 census data and put each zip code in its corresponding USPS district (which can be found here).  We also figured the percentages of the total population in each ZCTA living in urban and rural areas.

You can see a table with each ZCTA, the population data, and a link to a map showing the zip code area here.  A map of the country with some of this data is here. [Google Fusion Tables (with the maps) was shut down on Dec. 3, 2019. A spreadsheet version of the data is here, and a Carto map is here.].

 

 

A snapshot of the population

The table with Census data by zip code provides a helpful picture of the population of the country.  This table summarizes the Census data.

Rural Population in U.S.
% of total US pop. that is rural	Population	% of Total US Pop.	Rural Pop.	% of Pop that is rural	No. of zip code areas	% of total zip code areas
< 1 %	144,931,757	46.38%	0	0%	6,210	19%
1 – 20 %	77,807,793	24.90%	6,080,482	8%	3,708	11%
20 – 80 %	60,492,001	19.36%	24,645,872	41%	5,396	16%
80 – 99%	3,561,837	1.14%	3,236,829	91%	748	2%
100%	25,669,609	8.22%	25,669,609	100%	16,914	51%
Total	312,462,997	100.00%	59,632,792	19%	32,976	100%

Out of a total US population of 312 million, about 60 million people — around 20 percent — live in rural areas.  Of the 33,000 zip code areas in the country, 17,000 — more than half — are 100 percent rural, and several hundred more are at least 80 percent rural.  Almost 30 million people — half the rural population and almost 10 percent of the total US population — live in these rural and predominantly rural zip code areas.

About 145 million people — over 46 percent of the U.S. population — live in 6,210 zip code areas where the population is almost entirely urban (i.e., 99-100 percent).  Another 78 million live in 3,700 zip code areas that are predominantly urban (80 – 99 percent urban).

The remaining 5,400 zip codes areas are fairly mixed — between 20 and 80 percent rural — and 60 million people live in these areas.

These results show that for over 80 percent of the zip code areas, the distinction between “urban” and “rural” is fairly clear.

 

Combining the Census and performance reports

The next step in the analysis was to aggregate the Census population data by USPS district so that it would match up with the service performance reports.

The following table combines the results with the service performance report for Single-Piece First Class Mail, third quarter of 2015, year to date.  (The table is sorted from most to least rural, but you can see it on Google Docs where it can be sorted, downloaded, etc.)

USPS Service Performance for Single-Piece First Class Mail Q3 2015 TYD
District	% Rural	% On Time 2-day	% On Time 3-5-day		District	% Rural	% On Time 2-day	% On Time 3-5-day
Mississippi	51.0	95.9	78.3		Westchester	17.4	86.1	70.7
Appalachian	50.4	95.0	82.8		Atlanta	17.3	94.9	77.4
Dakotas	44.3	95.0	73.8		Northern New Jersey	16.7	90.8	73.9
Arkansas	43.8	94.7	80.2		Seattle	16.7	95.4	74.3
Greater Michigan	42.2	92.7	75.7		Colorado/Wyoming	16.0	93.1	70.0
Kentuckiana	42.1	94.3	78.6		Baltimore	15.8	91.2	72.2
Alabama	40.7	94.9	77.4		Caribbean	15.5	95.1	66.3
Albany	37.7	90.8	76.1		Greater Boston	15.5	90.2	71.2
Northern New England	36.8	92.7	71.5		Rio Grande	15.1	93.2	76.2
Greensboro	35.3	93.5	77.8		Salt Lake City	14.8	97.0	73.4
Hawkeye	35.0	95.4	79.1		Sacramento	14.1	93.3	78.3
Tennessee	34.1	95.4	77.8		Arizona	13.2	96.2	79.0
Alaska	34.0	97.2	81.4		Detroit	12.5	92.4	78.1
Oklahoma	33.6	96.1	81.9		South Jersey	11.4	95.2	77.8
Greater So Carolina	32.7	94.8	72.4		Houston	10.2	88.6	60.7
Mid-Carolinas	32.6	94.5	76.6		Northern Virginia	9.2	93.6	80.4
Mid-America	31.8	93.6	73.2		Central Illinois	9.0	94.2	77.6
Gulf Atlantic	31.3	95.7	78.0		Honolulu	8.1	97.4	76.3
Central Pennsylvania	30.0	93.9	75.4		Nevada Sierra	7.6	93.1	80.0
Central Plains	29.8	95.8	77.4		Suncoast	7.6	95.9	75.0
Northland	29.3	94.3	78.3		San Francisco	7.0	93.7	72.3
Western Pennsylvania	29.2	96.0	85.8		Capital	6.4	91.8	75.9
Gateway	27.2	91.9	73.7		Sierra Coastal	5.6	94.1	75.9
Louisiana	26.8	94.5	77.3		San Diego	4.9	92.3	73.0
Greater Indiana	26.0	93.7	78.6		Philadelphia Metropolitan	3.2	93.8	79.5
Western New York	25.0	95.5	81.8		Bay Valley	2.7	93.9	76.3
Richmond	24.5	90.8	75.5		Long Island	1.6	90.4	72.2
Ohio Valley	24.3	95.7	81.2		South Florida	1.3	94.8	67.4
Northern Ohio	21.2	94.6	80.0		Chicago	0.7	90.4	71.1
Lakeland	19.6	93.4	73.2		Los Angeles	0.2	86.7	63.8
Ft Worth	19.4	94.4	75.7		Santa Ana	0.2	92.6	70.9
Portland	19.3	95.8	75.1		New York	0.0	86.2	66.3
Dallas	19.0	93.0	72.8		Triboro	0.0	88.3	67.3
Connecticut Valley	18.3	90.8	73.4

Rural & urban service performance

The previous table shows that even the most rural districts have a large urban population, blurring the distinction between urban and rural.  In the most rural of all — the Mississippi and Appalachian districts — about half of the population lives in urban areas.

Thus, while it’s relatively easy to identify which zip code areas are rural or urban, it’s not so clear-cut when looking at districts.  Still, one can make some observations based on the relative percentage of people living in rural areas, as seen in the following table.

USPS Service Performance & Rural Population
Group	% of Pop. Rural	% on-time for 2-day	% on-time for 3-5 day
All 67 districts – average	20.5	93.5	75.4
All 67 districts – median	18.3	93.9	75.9
3 most urban districts	0	87.1	65.8
3 most rural districts	48.6	95.3	78.3
10 most urban districts	1.5	90.9	70.8
10 most rural districts	42.4	93.9	77.2
33 most urban districts	9.0	92.6	73.9
33 most rural districts	32.1	94.4	77.4
10 worst performing districts 3-5-day	6.2	90.2	67.5
10 best performing districts for 3-5 day	27.8	95.2	81.6

As the preliminary look at the performance reports had suggested, this table shows that the districts with a larger percentage of the population living in rural areas tended to experience better on-time performance than districts where a larger percent of the population live in urban areas.

For example, the ten most urban districts — i.e., those with the highest percentage of people living in urban areas or urban clusters — had an average 2-day on-time service performance of 91 percent, and a 3-5 day service performance of 71 percent.  In contrast, the ten most rural districts had an average 2-day performance of 94 percent, and a 3-5 day service performance of 77 percent.

The ten districts with the worst service performance for 3-5-day mail were predominantly urban (only 6 percent rural population on average).  Their average on-time performance was 67 percent.  The ten districts with the best 3-5-day service performance had an average rural population of 28 percent, and their average on-time performance was 82 percent.

In general, then, districts with larger a larger rural population (percentage-wise) had better on-time results than districts with larger urban populations.  Rural areas don’t seem to be suffering worse than urban areas.

Before jumping to any conclusions about these observations, however, it’s important to note several problems with the way the Postal Service does the measuring and reporting.  More than anything, this analysis simply highlights the problems identified by the GAO.

In fact, the key point in the GAO’s report is not that rural districts are suffering slow delivery times but that the measurement and reporting systems are done in such a way that it’s impossible to know how rural areas are doing.

According to the GAO, the main problems are incomplete measuring and the potential for sampling biases.  While not a major finding of the GAO report, another problem is the geographic level at which the analysis takes place.  Reporting at the district level simply does not provide enough information to evaluate service performance in rural areas.

 

Incomplete measurement

The GAO found that one of the main flaws with the service performance system is that the Postal Service doesn’t measure all the mail.  In fact, only about half the mail is actually tracked.

The Postal Service uses two primary methods for measuring delivery performance — barcoding for bulk mail (under the Full-Service Intelligent Mail program) and EXFC (External First Class Measurement System) for the rest of the mail.

The barcoding allows for a “census approach,” meaning that all the pieces are counted, as in a census.  (This is not a reference to the U.S. Census.)   In contrast, EXFC uses a sampling approach, meaning that only a small percentage of this mail is tracked, and based on the sample, estimates are made about the rest of the mail.

Because a lot of bulk mail isn’t barcoded and because single-piece is sampled, about 45 percent of market-dominant mail is not directly measured by the service performance reports.  The GAO found this to be a big problem.

“Incomplete measurement,” states the GAO, “poses the risk that measures of on-time performance are not representative, since performance may differ for mail included in the measurement, from mail that is not.”

“In general,” continues the GAO, “the risk that measurement is not representative is greater if mail not included in measurement may be systematically different than mail included in measurement.  In particular, if the unmeasured mail has different characteristics than the measured mail, and those characteristics are associated with the likelihood of on-time delivery, then the risk of a non-representative measurement is greater.”

 

Sampling problems

The Postal Service and the PRC told the GAO that they were confident that the methods used to report on service standards were sound, even if they were based on sampling rather than direct measurement.  It should also be noted that the USPS Office of Inspector General once looked at the sampling issue and found that the size of the sample might actually be reduced, which would save costs for the Postal Service.

The clear implication of the GAO’s report is that sampling may not be fully capturing what’s happening in predominantly rural areas.  For example, since barcoding is done mostly by large volume mailers, their mail is more likely to have their on-time delivery measured directly.

Other types of mail may also be under-represented in the sampling.  As the GAO points out, the EXFC testing only looks at mail pieces deposited in blue collection boxes and office building lobby chutes and doesn’t include mail picked up from customer mailboxes or accepted at postal retail counters.  That leaves out a lot of mail.

As the GAO notes, “According to a 2013 USPS study, 33 percent of single-piece First-Class Mail is deposited in blue collection boxes, 29 percent is accepted at postal retail windows and docks, and 38 percent is picked up by carriers from customer mail receptacles.”

Along these same lines, the OIG report on EXFC came to a similar conclusion: “The EXFC program only measures single-piece First-Class Mail originating from about 34 percent of potential collection points and does not represent customer expectations.  We do not believe analyzing test piece mail originating from only about 34 percent of potential collection points meets the EXFC‘s program objective of measuring service performance. “

If people in rural areas tend to leave mail for pickup in their mailboxes or at the post office (as opposed to blue boxes and office chutes), then it’s possible that their mail is not being measured to the same extent as urban mailers.  If this mail is not being counted, it adds to concern that the performance reports are inaccurate.  Perhaps rural users are getting slower service, and it’s just not being measured.

 

Gaming the system

Another problem, but one not discussed in the GAO report, is gaming the system, which gets a considerable amount of attention in the OIG report on EXFC.  Here’s how the testing works.

The Postal Service contracts with an external contractor — IBM — to conduct a sampling system that measures the time it takes from when mail is deposited into a collection box or lobby chute until its delivery to a home or business.  A group of about 1,300 “droppers” sends the mail, and a group of about 17,000 “reporters” receives the mail.  Everyone records when the mail was sent and received.

Almost 3 million pieces of mail are tracked this way, and using that sample, estimates are derived about service performance for billions of pieces of mail that aren’t tracked census-style.

According to the OIG, “EXFC is vulnerable to ‘gaming‘ as management and staff often give potential test pieces preferential treatment to raise scores.  For example, personnel take extraordinary and costly measures to separate and deliver possible test pieces.”

The system also gets gamed when Postal Service employees themselves participate in the EXFC testing process by inducting mail and reporting mail delivery times.  This could be avoided by modifying the contractor‘s tester selection procedures to prevent Postal Service employees and their family members from participating in the EXFC program, but as of 2012, this had not been done.

For reasons like these, some of the Postal Service’s customers told the OIG that they had more confidence in the reliability of data about barcoded mail.  “Manual processes and sampled data provide opportunities for special treatment of mail and ‘gaming‘ of scores.”

 

Going deeper into the weeds

Although the GAO doesn’t get into it in much detail, another issue with the performance reports is that they are done at the national, area, and district level, rather than getting down to the level of 3 and 5 digit zip codes.  Even at the district level, it’s hard to distinguish between rural and urban areas.  Every district has both.

As the Postal Service told the GAO, “On-time delivery performance information at the district level cannot inform stakeholders on delivery performance in rural areas since each of USPS’s 64 districts in the continental United States contains at least one core area with a population over 10,000 and thus is not entirely rural.”

Although this is not an explicit recommendation of the GAO report, it seems that a robust sampling and reporting methodology would include performance results not on a district-by-district level but on a zip-by-zip level, where the difference between rural and urban is more meaningful.

At the least, the Postal Service could report on performance at the level of 3-digit zip code prefixes.  As shown by this table (on Google Docs), which aggregates the population data by 3-digit prefixes, of the 892 3-digit areas, 6 are 100 percent rural; 31 are at least 80 percent rural; and 72 are at least 70 percent rural.  Not surprisingly, this view provides a more nuanced look at which areas of the country are rural than by looking at the district level.

The Postal Service may say that reporting at the 3 or 5 digit level would be cost-prohibitive, but it might not be so difficult.  The Postal Service may already have the data it needs.

Postal Service officials told the GAO that “its reports are generated at the national, area, and district level for these [reporting] purposes, but are not routinely further disaggregated on the basis of whether particular districts or zip Codes are rural, suburban or urban in nature.”

 

A zip-by-zip approach

The Postal Service may not disaggregate the data by zip code for reporting purposes, but it’s likely that the Postal Service already has performance data on a zip-by-zip basis.

For barcoded mail, the barcode includes fields for the origin and the destination zip code (as explained on this fact sheet).   Not all mailers include this information in their barcodes, but they are encouraged to do so in order to maximize the discounts they receive.

For the mail sampled using EXFC, the Postal Service probably has zip code level data as well.  According to a description of EXFC in the OIG report, the volunteer group “mails test pieces from nearly all zip codes across the country.”  Presumably, then, the Postal Service knows where the test mail originated and destinated.  It probably has access to zip-by-zip performance for this mail.

If the Postal Service does have performance data at the zip code level, it could correlate the data with the population data for rural and urban for each zip code as drawn from the Census website, just as we did for the analysis of districts.  This would allow for a more nuanced analysis of service performance.

Such an approach could be even more helpful than enlarging the size of the EXFC sample or trying to measure the mail census-style.  At the least, it would help determine if the mail being sampled fairly represents rural populations.

A zip-by-zip approach would also address the problem of defining “urban” and “rural.”  While each district is a mix of urban and rural, most 5-digit zip code areas are clearly one or the other, and it’s much easier to differentiate 3-digit areas than districts.

Plus, it’s not entirely necessary to decide if a particular zip code area is classified as one or the other.  Instead, one can make statistical correlations between the extent to which the population is rural (percentage-wise) and the levels of on-time performance, as we did with the analysis of districts.

In the end, such an approach might produce the same results as the analysis at the district level.  One might find that people in rural zip code areas are getting the same level of on-time performance as people in urban areas, or maybe even better.

But until the Postal Service and the PRC get a little deeper into the weeds and start reporting on service performance at the zip-code level, it’s anyone’s guess how rural areas are doing, and assurances that the measuring systems are working properly will not convince some members of Congress.

(Image credits: Census sign & Blue Box; “The Census Taker” by Norman Rockwell; Census map of urban & rural US)