Web surveys create potential for confusion

Radio talk show host Dori Monson knows the Internet surveys he reads on air are bogus. But he doesn't care.

Monson says his main goal is to generate debate.

"The statistical validity of the numbers is not as important as the debate it will inspire," says Monson, who hosts a daily show on KIRO-AM in Seattle.

Professional pollsters worry that the proliferation of unscientific online surveys, in which people with vested interests can easily tilt the scales, mislead the public.

And people like Monson show how such pseudo-surveys can be misused, despite efforts by some news organizations to distinguish online tallies from legitimate surveys that are based on random samples.

Warren Mitofsky, who used to direct polls for CBS News, says online surveys at news sites -- even with disclaimers -- are irresponsible.

"All they are doing is playing games," Mitofsky said. "If something comes from a reputable news organization, as a member of the public I would expect it to be reliable."

Recent surveys

In recent weeks, online surveys have asked visitors to rate President Bush's State of the Union speech and opine on whether the tattered American flag from ground zero was too political for the Olympics.

During the Super Bowl, Internet users were asked about coaches' strategies and their picks for most valuable player.

Television networks often promote the surveys on air, and CNN's Wolf Blitzer read the results of one on Monday, along with a disclaimer that "this poll is not scientific."

"We call it a quick vote and not a poll, so it's not confused with a scientific sampling," said Mitch Gelman, executive producer of CNN's Web site.

MSNBC has a 900-word explanation about the difference, though users must click on a link to get it. A few hundred do so each day, out of thousands who participate.

Savvy decision-makers are not likely to confuse Web surveys for sound research, but everyday users generally don't think about the difference, said Sreenath Sreenivasan, a journalism professor at Columbia University.

The harm comes when users start shaping their views based on what they think everyone else is thinking, Sreenivasan said. Even on seemingly innocent questions like favorite car color, he said, a buyer may choose a color believing it will have better resale value.

Self-selection is the chief problem with online surveys. Because users choose to participate, the surveys attract people who have time, are comfortable with the Net or are strongly opinionated.

Another major limitation of Internet surveys rests with the fact that 46 percent of Americans don't use the Internet, while only about 5 percent of households lack phones. And Internet users don't visit every site, so at most a Web survey can reflect what users of that particular site think.

Consider the results of one online survey at Time magazine's Web site. Visitors were asked to grade Bush's job as president. Nearly 80 percent chose either A or F, suggesting heavy participation by partisans.

At ZDNet's U.K. Web site in December, participants favored Sun Microsystems' Java programming language over Microsoft's .NET for building Web services. By early January, .NET took over as the leader.

ZDNet found that a high percentage of votes came from Microsoft employees. Logs showed that many responded directly after receiving an e-mail that said, "PLEASE STOP AND VOTE FOR .NET!"

Ballot stuffing and lobbying are quite common, particularly for heated issues like abortion and Mideast politics. Though many sites attempt to block repeat votes, they can't always stop them.

The Rev. Gregory Dell, a Chicago pastor who supports letting gays adopt children, recently urged his congregation and other like-minded pastors to vote at the local NBC site on the issue.

"People feel an impact when any poll results are released," he said. "Even with the caveat that's often declared, nevertheless it leaves a perceptual impact."

Results inconclusive

The results of online surveys sometimes mirror those of telephone polls, but it's impossible to know when a Web survey will be right or wrong.

Still, news organizations justify them as something fun and unique to the Internet. They say they need interactive features to keep visitors glued to their sites so they are more attractive to advertisers.

Online polls seem like an easy answer.

"Online polls are really more like bells and whistles," said Richard Stengel, managing editor of Time.com. "They are ornaments on the tree."

Steve Jones, executive producer for ABC News' site, said Web surveys are "100 percent credible and accurate" as presented -- it's only a problem when readers extrapolate too much meaning from them.

ABC, among others, limits online surveys to soft subjects, like what visitors consider the movie with the best Tom Cruise kiss.

The New York Times' site uses them primarily in its magazine section. For hard-core political issues, the site won't present results, but will instead show users how they compare with legitimate polls.

The Washington Post's site dropped political questions about four years ago after results kept swinging back and forth, but it still uses online surveys for sports.

"Questions like, 'Is this the greatest Super Bowl?' -- the future of the world hardly depends on the outcome," said Douglas Feaver, the site's executive editor.