Last week I wrote a post about the Postal Service’s new employee survey, Postal Pulse. The Pulse replaces the Voice of the Employee (VOE), which the Postal Service has been using for the past 17 years. The Pulse is now being administered to over 600,000 postal workers, and the Postmaster General is encouraging everyone to fill it out.
My criticisms focused on the fact that the Pulse seemed inferior to the VOE. The new survey omits many questions in the VOE about workplace diversity and discrimination, safety, and so on, which seemed worthwhile. The article also described the Pulse questions as being “touchy-feely” and intended to encourage “sensitive, New Age conversations” between management and workers.
Little did I know that I was criticizing the most famous “employee engagement” survey in the world, the Gallup Q12. Rather than creating its own survey for postal workers, the Postal Service has simply contracted with Gallup to administer the Q12 survey.
The Postal Pulse survey is here; the Q12 survey is here. As you’ll see in comparing them, the questions are the same, almost word for word. The only difference is that the Pulse adds an introductory question about how satisfied workers are at the Postal Service. It’s No. 0 so as not to throw off the numbering of the other 12 questions, which correspond, one by one, to the 12 questions on the Gallup Q12.
Since it was developed in the late 1990s, Gallup’s Q12 Employee Engagement Survey has been administered to more than 25 million employees in 189 different countries and 69 languages for use by several hundred organizations. It’s considered the “gold standard” for employee engagement surveys.
Credit for the discovery that the Pulse is simply the Q12 by another name goes to PostalMag.com, which wrote about the scoop in this post.
Now that PostalMag has pointed it out, though, we should have probably noticed that there’s a footnote in small print on the Postal Pulse survey that says, “Gallup and Q12 are trademarks of Gallup.” Who knew?
“Developing” Postal Pulse
It’s not clear why the Postal Service hasn’t come right out and said it is using the Gallup Q12. It’s a highly respected survey with a lot of social science behind it, and many people would probably be happy to learn that the Postal Service had shifted over to such a well-known product.
For some reason, though, the Postal Service has made it seem as if Postal Pulse is something unique to the USPS and that it helped develop the survey.
For example, in her recent video talk to employees, as PostalMag.com observed, the new Postmaster General Megan Brennan clearly indicated that the Postal Service had helped create the Pulse. Here’s what she said:
“What’s different about this survey [compared to the VOE] is that it was designed based on your feedback, to make something simpler, more secure, and most important, more actionable.”
It’s hard to see how postal workers could have had any impact on the design of Postal Pulse, considering that the Q12 was created over 15 years ago.
The PMG went on to say, “To develop this survey, we partnered with the Gallup organization, a recognized world leader is research and employee engagement, to design something that will collect your views about our workplace culture.”
That too is a bit misleading. The Postal Service didn’t “develop” the survey and it didn’t “partner” with Gallup to “design” anything. The Q12 had already been designed, and the only “partnering” that’s going on is that the Postal Service is paying a hefty fee to Gallup for administering the survey and analyzing the results.
A USPS press release about the Pulse also made it seem as if the Postal Service helped create the new survey. It quotes Chief Human Resources Officer Jeff Williamson saying, “The streamlined questions will yield better data that supervisors and managers can use to put employees’ feedback into action.”
That makes it seem as if the Postal Service “streamlined” the survey to improve it, but that’s obviously not the case. The Postal Service didn’t put any thought into streamlining the new survey. The only thought that went into it was the decision to hire Gallup and use the Q12.
While the Postal Service could have developed its own unique survey based on the type of questions in the Q12, that would have defeated one of the main advantages of using the boilerplate Gallup poll. It was very important that the Postal Service not change the wording of the questions.
The Q12 database, with its millions of responses, is the largest employee benchmark around, so sticking with the exact same questions allows Gallup and the Postal Service to compare the results for postal workers with the results for other organizations and the benchmark.
This also means that when postal workers fill out a Postal Pulse survey, they aren’t just giving feedback to postal management. They are also adding their information to the Gallup database. The PMG has promised to keep each individual’s survey confidential from supervisors, but she didn’t say anything about not sharing the results with Gallup.
Although the Postal Service can’t take any credit for the questions in Postal Pulse, someone decided it might be a good idea to trademark the name. As PostMag points out, on February 6th, 2015, the Postal Service filed a trademark application, under goods and services, for “conducting employee surveys for others for purposes of improving employee performance and morale.”
Not that it took a lot of work to come up with the name Postal Pulse. The Gallup website page hawking the Q12 says, “Take the pulse of your workforce with Gallup’s simple 5 minute survey and ask the 12 questions that tie directly to your most important business outcomes.”
Costs and contract
Back in 2007, the Postal Service posted a solicitation request on FedBizOpps.gov looking for “suppliers who can develop, implement, manage, and analyze information for a national Employee Engagement Survey.” (The full notice is archived on Postal Reporter, here.)
At that time, the Postal Service had already been doing the VOE for several years, so it’s not clear what the bid notice was about or if a change took place at that time. In any case, it’s likely that the VOE was done under contract with an outside supplier. Aside from Gallup, there are many companies that do VOE surveys, including Converge Consulting and People Metrics.
There doesn’t seem to be a comparable solicitation anytime recently, so it’s hard to know what kind of bidding process, if any, the Postal Service went through in selecting Gallup to do the new survey or why a change was made.
However it came about, the Postal Service’s contract with Gallup is undoubtedly confidential, and one can only guess how much the survey is costing.
According to the Gallup website about the Q12, “Gallup standard programs range from $5 – $50 per employee depending upon the unique demands of your organization, with a survey only version available for small-to-medium sized enterprises at $15 per employee.”
According to Hazen Witemeyer, who completed a Ph.D. dissertation at Georgia State on employee-engagement surveys, Gallup’s full-service approach can cost $23 to $27 per person polled. It’s not clear if the price is based on how many employees receive the survey or on how many actually complete it.
The Postal Service says the survey will be given twice a year to all of its employees, career and non-career. That’s over 600,000 people to be polled.
Considering the huge size of the postal workforce, one can assume the USPS got a good deal. But even at the lowest price of $5 per person, with the survey happening twice a year, the total would come to $6 million. If the Postal Service is paying $25 for every employee, whether or not they fill out the survey, the cost could be as much as $30 million.
That may seem like a lot of money to learn how workers feel about their jobs, but Gallup’s data analysis shows that “companies with highly engaged workforces outperform their peers by 147% in earnings per share.” They also do better in terms of safety incidents, turnover, and absenteeism, which can save huge amounts of money. The Hay Group, which provides its own surveys like the Q12, says high levels of engagement can boost revenue growth by up to two and a half times.
Given the promise of such improvements in productivity and profits, companies are willing to pay big bucks for surveys like the Q12. The surveys are thus big business. According to one source, almost $1 billion worth of engagement surveys are sold in North America each year.
Controversy and criticism
While the Q12 and similar surveys are extremely popular, they have come in for a lot of criticism.
One of the main critics is Peter Hutton, an HR expert who wrote What Are Your Staff Trying to Tell You?, published in 2008. Hutton concluded that whatever it was employees were trying to tell managers, it wasn’t going to show up in the survey.
One problem Hutton points to is that the Q12 questions are all in the agree/disagree format. (It’s called the Likert scale.) According to an interview with Hutton, agree/disagree questions are the weakest type. As Hutton explains, “Most things you want to ask about a company do not fit this way of asking.”
“There is no other research that just uses agree/disagree questions and then makes such large claims from it,” says Hutton. “It’s a joke.”
Another criticism made by Hutton is that using standardized surveys like the Q12 can be less effective than a survey individualized for a company. “But when you take an off-the-shelf product,” says Hutton, “it’s like admitting you’ve booted out your own company visions.”
A more recent study about engagement surveys was conducted in 2013 by Robert Gerst of Converge Consulting Group, one of the companies that does its own VOE survey. Entitled “Understanding Employee Engagement and Trust: The New Math of Engagement Surveys,” the report says this:
“The dirty little secret of employee engagement surveys is that they’re largely junk science — placing the marketing objective of telling and selling a good story above the practical and ethical objective of telling the truth.”
According to Gerst, the main problem is that interpretation of the results is basically a pseudoscience. He describes various ways in which the feedback gets corrupted, such as assuming complex concepts like engagement can be reduced to a single numerical index.
Still another criticism, as noted in this article, is that while the surveys can help identify weaknesses in an organization’s work culture, they don’t offer solutions or tangible fixes.
In a good post about the surveys on his “Changing Winds” blog about management issues, James Taggart puts it this way:
“The definition of insanity,” writes Taggart, “has been described as expecting a different result while doing the same thing over and over again. This describes employee engagement surveys.”
Taggart goes on to issue this warning:
“Don’t get caught up in the hyperbole of employee engagement. As with past fads (e.g., Business Process Re-engineering, Total Quality Circles, and the Learning Organization), the employee engagement fad is being milked for all its worth by consultants who hope to make a few bucks from it before attention turns to the next fad. And of special importance is understanding that mouthing the words ‘employee engagement’ by management is typically a tactic to deflect attention, however briefly, from the substantive issues facing organizations.”
A best friend at work
My own criticism of the Postal Pulse (before I learned it was the Q12) was that it seemed to be all about self-esteem issues, personal growth, and getting respect — not about what might be wrong with the workplace itself.
Of course, now it’s clear that this is exactly what the survey is supposed to be about. As Gallup explains, the Q12 “measures employee’s emotional engagement, which ties directly to their level of discretionary effort — their willingness to go the extra mile for their company.”
That’s why the survey has questions like No. 10, which asks workers to agree or disagree with the statement, “I have a best friend at work.”
Postal workers have been wondering why the Postal Service is asking them this question. It seems kind of ridiculous.
In fact, the question gets a lot of attention in media commentary on the Q12, and this article in Forbes explains why the question is there: A lot of data shows that “friendship is more important than pay or benefits, and strongly correlates to productivity, safety, customer loyalty and profitability.”
That, I guess, is the answer to the question, What are friends for? So you’ll work harder.
(Cartoon credits: Dilbert; chicken survey; Dilbert; Engels – walking off cliff; Dilbert)
