There’s an on-going war between surveys in so far as the forthcoming 2010 election is concerned. So heated are the debates between ardent supporters of candidates, especially among supporters of Noynoy Aquino and Gibo Teodoro, that one commenter likened it to a boxing match. Aquino is leading in the Pulse Asia’s October survey and the Social Weather Station’s 3rd quarter and 4th quarter surveys. Teodoro, on the other hand, is leading in smaller scale surveys like Manila Bulletin and Facebook surveys as well as the UP mock polls.

These results behoove us to ask how the surveys were carried out, so we can judge for ourselves whether they are valid, accurate and credible. Surveys involve many statistical research processes that may be too technical for the average Filipino. However, we have to get into the nitty-gritty of these opinion polls if we are to achieve a greater degree of education and maturity as voters.

Survey results should not be taken at face value because biases can be introduced in the data gathering process, which render the results inaccurate or even invalid. The order of answer choices is one source of bias. I have observed this in telephone interviews where busy respondents can’t wait for the interview to be over so that they cut the interviewer midway, opt for the first choice and do not wait anymore for the rest to be read out. In the fourth quarter SWS survey,

The question asked for the Presidential preferences was, “Sa mga sumusunod na pangalan sa listahang ito, sino po ang malamang ninyong iboboto bilang PRESIDENTE ng Pilipinas, kung ang eleksyon ay gaganapin ngayon?” [Among the names found in this list, who will you probably vote for as PRESIDENT of the Philippines, if elections were held today?]. The respondents were furnished with a BusinessWorld-approved list of 11 candidates to choose from, and gave their answers orally.

All lists were in alphabetical order, and included nicknames, as practiced by the Commission on Elections on election day.

Randomization is prescribed in the research industry to minimize response bias. In the SWS survey, the fact that the answer choices were not randomized but alphabetized is already one reason we should be cautious about the results. I wonder if the “as practiced by the Commission on Elections on election day” was added as some sort of a disclaimer?

Another source of bias is sampling. An accurate survey should show a good representation of the entire population. This is achieved by getting a sample that is a cross-section of the voting population, much like the cross section of an orange, with each segment representing each attribute of the population, e.g., socio-economic classes, different geographical areas like provinces. We ought to evaluate the surveys along these lines. In the SWS Dec. survey,

The Fourth Quarter of 2009 Social Weather Survey was conducted from December 5-10, 2009 using face-to-face interviews of 2,100 adults, for an error margin of ±2.2%. The national estimate was obtained by using area estimates weighted by National Statistics Office medium-population projections for 2009.

and Pulse Asia’s October survey is based on

… a multistage probability sample of 1,800 representative adults 18 years old and above, Pulse Asia’s nationwide survey has a +/- 2% error margin at the 95% confidence level.  Subnational estimates for the geographic areas covered in the survey have the following error margins at 95% confidence level: +/- 6% for Metro Manila, +/-4% for the rest of Luzon and +/- 5% for each of Visayas and Mindanao. Face-to-face field interviews for this project were conducted from October 22 to 30, 2009.  (Those interested in further technical details concerning the surveys’ questionnaires and sampling design may request Pulse Asia in writing for fuller details, including copies of the pre-tested questions actually used.)

Somehow this background information on the surveys does not offer much as to how their respective samples are actually distributed across socio-economic classes, so we cannot make a complete judgment as to whether classes A, B, C, D and E are proportionately represented. As regards the Facebook survey, we can take a guess at who the respondents are, and based on that, we may conclude that the composition of the respondents is not a good representative of the voting population.

Then there’s the on-going popularity contest between Aquino and Teodoro on Facebook, which is debunked by Paul Farol here. (While posting this, I came across a post at Better Philippines that Farol earned the ire of Noynoy supporters and has been removed from Asian Correspondent. So in case the link to his post leads you nowhere, this explains it.)

Interest groups are already exploiting these opinion polls to maximize the bandwagon effect. For instance, one news article reads: Noynoy leads in Makati Business Club Survey, and it goes on that:

Among presidential aspirants, 61 percent of members chose Aquino, followed by former defense chief Gilberto Teodoro with 14 percent, and Sen. Manuel Villar, seven percent.

What it doesn’t mention early on in the report (but did in the last paragraph) is that

Close to 13 percent of the MBC’s members participated in the mailed survey.

Only 13% of the Club’s members participated in the survey! While reporting that “61% members chose Aquino…” is accurate as far as those who responded are concerned, it does not speak for the 87% who did not participate in the survey. Now, that says a lot.

Lauren Dado of Philippine Online Chronicles and one blog at Philippine Commentary virtually took apart the method employed by the SWS in their 3^rd quarter presidential survey. Noynoy’s 60% rating based on this survey is repeatedly spread around by word of mouth, misleading many, especially those who don’t bother to check the details, that respondents were allowed to give up to three answers. Anyone who simply took the 60% at face value would think that 60% of the respondents favored Noynoy, which is an erroneous interpretation.

Surveys always figure much in elections. The foregoing, however, is proof that interest groups have the survey results at their disposal to further their, well, interests. What we are seeing are groups capitalizing on surveys to induce a bandwagon effect and exploit the fact that maraming mga Pinoy ang ayaw nakakantiyawan, ayaw maging talunan. It is these people that these groups target to take on board. However, it is a great disservice to the voting population because, as they hop into the proverbial wagon and halfheartedly or blindly vote for the most popular candidate, the voters forgo their opportunity to make an informed choice. In the end, it hampers the much-needed maturation of Filipino voters.

Related links:
Why is Noynoy losing support?
Tatad questions reliability of surveys

Filed under: In the news, January 2009, Local Politics | You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.