Last summer the Postal Service hired Opinion Research Corporation (ORC) to do market research on how customers might respond to changes in service standards (slowing down the mail), as well as plans to close thousands of post offices, eliminate Saturday delivery, and seek legislative reforms on pensions and the retiree health care fund. The purpose of the research was to provide data on which to base estimates of potential losses in mail volumes and revenue.
In her testimony before the Postal Regulatory Commission (PRC) for the Network Rationalization plan to consolidate over 200 mail processing plants, ORC’s Rebecca Elmore-Yalch described the two components of the research. The qualitative research, conducted in August 2011, evaluated customer attitudes, while the quantitative research, conducted in October-November, focused on exactly how much less business mailers might do with the Postal Service if the mail slowed down.
Although the witnesses for the Postal Service were reluctant to mention it, we’ve learned from testimony that there was another quantitative study, done back in August or September, at about the same time the qualitative research was being done. The Postal Service now says this phase-1 quantitative research was “abandoned” before it was completed, “with the result that no analysis of its preliminary results was pursued.” The Postal Service paid ORC over $430,000 to do the work, but it never analyzed the results.
The Postal Service has given the PRC the data for both phases of the quantitative research, but they are classified as “non-public” library references. The Postal Service testimony (USPS-T-12, p. 22) includes calculations of lost volume and revenue for the phase-2 quantitative study, but not for phase-1.
The Postal Service says that because it didn’t complete the study there are no results to look it. But when the existence of the other market research was first revealed in testimony by Gregory Whiteman, Manager of Market Research at the Postal Service, Whiteman stated, “In short order, the Postal Service plans to file two documents summarizing this research and its results.”
The PRC docket contains the market-research instruments — the concept statements and questionnaires — that were used for all three studies (the qualitative and both quantitative). It’s worth taking a closer look to see how they were constructed and how they were changed for the phase-2 survey. From the looks of things, it seems that the second time around, the Postal Service wanted to elicit responses that would result in less significant impacts on mail volumes and revenues.
The phase-1 concept statement
In all three studies, participants were read (or asked to read) a “concept statement” before responding to the questions. (The concept statement for phase-1 is here, and for phase-2, here. The phase-1 statement comes from the materials submitted to the PRC on March 9 as Library Reference 70; see Questionnaire, p. 11. The phase-2 statement comes from Elmore-Yalch’s testimony, USPS T-11, p. 82, 83, 88.)
For both the qualitative research and the phase-1 quantitative research, the concept statement describes “the fiscal situation facing the Postal Service and proposed changes to the network, legislation, and First-Class Mail service standards proposed to address the situation.”
This concept statement reads as follows: “As a result of declining mail volume, the cost to continue providing this level of service is becoming unsustainable, contributing to major budget deficits for the Postal Service. In the past two years, the Postal Service has had budget deficits of over $8 billion and expects to have a similar budget deficit this next year.”
The statement proceeds to state that the Postal Service “is exploring several options” to deal with the deficit:
- Legislative reform to change government requirements to pre-pay health and pension benefits
- Eliminating Saturday mail delivery to homes and businesses
- Closing many small post offices while shifting retail access to alternative locations and channels
The statement then describes the changes in service standards that would occur with the Network Rationalization plan — next day delivery will become two-day, some two-day will become three, and so on.
The phase-1 concept statement thus puts the proposed changes in service-standards in the context of other proposed changes — five-day delivery, closing post offices, and legislation about pensions and health care.
The Postal Service has made public the ORC report about the qualitative research, which summarizes and quotes the comments of many participants. The responses show that customers were clearly sensitive to the problem of combined effects, and many referred to the various options in their responses.
The phase-2 concept statement
The phase-2 quantitative survey contains a new concept statement. This one does not mention closing post offices, eliminating Saturday delivery, or legislative reform. This statement goes directly into a discussion of the change in service standards, and it provides much more detail about these changes than the concept statement for the phase-1 research.
This expanded discussion of the change in service standards includes passages that make the new processing network sound as if it will actually improve service for some customers. For example, while the phase-1 concept statement made no mention of it, the phase-2 concept statement indicates that some mail may still qualify for next-day delivery:
“For First Class mailers who presort their volume to the destinating service area of the plant and enter the mail prior to a Critical Entry Time of 8 am, the First-Class Mail will be delivered the next day. For First Class mailers who presort their volumes to the destinating service area of the plant and palletize the mail by Five-digit ZIP Codes and also enter the First-Class Mail prior to a Critical Entry Time of 12 noon, the First-Class Mail will be delivered the next day.” (USPS T-11, p. 100)
Both the phase-1 and phase-2 concept statements include a paragraph stating, “Businesses using bulk First Class, Standard, or Periodical Mail may have access to fewer locations accepting this mail and potentially result in a need to transport this mail to a location different from the one they are currently using.”
The phase-2 concept statement follows that paragraph with a new passage: “However, there is also the potential that these mailers may be able to achieve improved transportation efficiencies since the service areas of these facilities may be larger than they are currently. For example, if a mailer currently sends mail to two facilities which are consolidated to one, this can allow the mailer to prepare a larger mailing for deposit at the consolidated site, resulting in a reduction in the required transportation.” (USPS T-11, p. 100)
The phase-2 concept statement thus omits references to other initiatives — closing post offices and ending Saturday delivery — which might negatively impact mailing behaviors, and it spins the changes in service standards to make them sound better — or at least less negative — than in the first survey’s concept statement.
The screening question
Both the phase-1 and phase-2 quantitative research involved asking customers a series of questions about their mailing behavior, before and after the change in service standards. But before getting into the nuts-and-bolts questions, there’s a screening question posed in phase-2 that does not appear in phase-1:
“[NATIONAL SAMPLE ONLY] Have you participated in a survey for the United State Postal Service in the last three months?” (p. 90).
If the answer was yes, the interviewer asked to speak with someone else at the company or terminated the interview.
As noted in the bracketed directions, this question was asked only of the National Account participants. These are customers who do more than $10 million a year in business with the Postal Service. There are about 236 of them.
Unless there were other surveys conducted with the National Accounts during August to October, it would appear that the Postal Service did not want those who had participated in the phase-1 quantitative survey to participate in phase-2. We don’t yet know why the Postal Service did not want customers participating in both surveys. Was there a concern that it would skew their responses? Or was the Postal Service trying to avoid alerting their National Accounts to the fact it was doing a second-round quantitative survey?
According to Elmore-Yalch’s testimony, since these accounts are so important, “every effort was made to maximize use of this sample” and to “maximize the number of completed surveys” (p. 25). However, for the phase-2 survey, only 26 National Accounts participated, in contrast to the 59 that participated in the Five-Day Delivery market research, also conducted by ORC.
Asked by the APWU why so many National Accounts did not participate in the survey, Elmore-Yalch explained that “the primary reasons given for not completing the survey were (1) a company policy against completing survey research or (2) no time then available to complete survey.” She didn’t mention the screening question or indicate how many interviews, if any, were terminated when the customer answered yes.
Questions on the likelihood of changing volume
In both phases of the quantitative survey, one of the first questions customers were asked involved the likelihood that the changes in service standards would affect the number of pieces of mail they sent. There’s a slight but perhaps significant change in the way that question is phrased.
For phase-1, the question reads, “Assuming that the changes to First Class Mail had been in place during the past 12 months, what is the likelihood that this change would have caused your organization to modify the number of individual pieces of mail your organization sent by any means. Please answer using a scale from 0 to 10, where ‘10’ means extremely likely and ‘0’ means extremely unlikely.”
For phase-2, the question reads, “For these next questions, please answer based on the assumption that the First- Class Mail service standards I have just described will be in place in 2012. You indicated that the total volume of mail that you would mail in 2012 through the Postal Service would be [RESTORE TOTAL NUMBER OF PIECES (TOTAL VOLUME BEFORE)].”
The phase-1 phrasing assumes that the changes have been in place for the past 12 months, while phase-2 says the changes will take place in the future. Perhaps a market-survey psychologist can explain how the two variations of this question might elicit different responses.
Questions on changes “solely” attributable to service standards
The phase-1 survey asks customers several questions about their current mailing habits — what percentage of their communications they mail single-piece First-Class, presort First-Class, Priority, and Express — and so on. The survey then goes into the key questions about how the customer’s behavior might change with the new service standards.
For the 15 or so questions on this topic in the phase-2 survey, eight ask the respondents to focus solely on the change in service standards. For example:
“You indicated that based on the First-Class Mail service standards I described the total number of documents you would mail using the U.S. Postal Service in 2012 would [DECREASE / INCREASE] by [RESTORE DIFFERENCE DOCUMENTS] pieces. What percentage of this [DECREASE / INCREASE] is solely because of the First-Class Mail service standards that I described?” (Testimony, p. 108)
In her testimony about the phase-2 survey, Elmore-Yalch calls attention to the fact that the customers were asked to describe potential changes in their mailing behavior that were “solely attributable to changes in the First-Class Mail service standards.” Similarly, in the table on page 49 of her testimony, which illustrates how survey responses were converted into volume estimates, one of the factors is “% of Increase / Decrease in Volume Solely Attributable to Change to FCM Standards.”
The questions in phase-2 thus went out of their way to filter out the effects of closing post offices and ending Saturday delivery, even though they weren’t even mentioned in the concept statement. As suggested by several interrogatories on the subject, this presumably was necessary because there was so much news about these other factors in the media, and the Postal Service didn’t want them to influence customer responses in the survey.
Questions for the witnesses
Next week, the participants in the Advisory Opinion process will have an opportunity to cross-examine the Postal Service witnesses and ask questions about the two phases of the market research. Perhaps someone will ask why witness Whiteman indicated that the results of the phase-1 research would be forthcoming but we’re now told the research was abandoned before the results were calculated. Perhaps someone will ask why the phase-2 survey was crafted in ways that isolated impacts due “solely” to service standard changes and deleted all references to closing post offices and eliminating Saturday delivery. Perhaps someone will ask why the phase-2 survey used a new concept statement that made it appear that the changes in the processing network might actually improve service for some mailers. And perhaps someone will ask who made the decision to abandon the phase-1 research and to try again with a second round of research.
It’s sure looking as if someone at the Postal Service didn’t like the results of the phase-1 survey and decided they could do better the second time around. But perhaps there’s a more innocent explanation. It will be interesting to hear what it is.
