You Mix Fact and Opinion – One of the Top 10 Mistakes Survey Writers Make

Editor’s Note: This is the last in a series of posts on common Survey writing mistakes. Click here to see the previous item!

MISTAKE #10: YOU MIX FACT AND OPINION

Face it – there is a difference between the way people THINK things are and how they REALLY are (theoretical physicists might take this a step further and say there are many ways things are, but here in the real world we prefer to believe there is an objective reality). Moreover, there are differences between the way we think things were in the past and the way they really were. Let’s call this the perception vs. reality conundrum. Within the context of Surveys this presents a whole bunch of challenges. For example, consider the following question:

How is your company performing this year?

Is this question intended to elicit factual information on some objective metric, i.e. how revenue is growing or shrinking? Or is it intended to yield respondents’ opinions on how the current state of their business? Moreover, in either case, does the potential respondent even know? In other words, will their answer be accurate or simply reflect their perception of how things are going? I consider this to be probably the single biggest mistake that Survey writers make. It has the greatest potential to yield poor, misleading, or confusing results. What’s worse, it can be very difficult to detect problems introduced by mixing fact and opinion, since the person who reads the Survey results is not likely to correctly identify the issues with a question like the one above. Do not be tempted to believe that this is the same thing as vague language. Yes, the question above is exceedingly vague, but the more immediate problem is that it does not force the respondent to EITHER answer factually OR express an opinion. Depending on the context, either fact or opinion could be valuable and appropriate, but the two should never be confused, mixed, or conflated. It should be screamingly obvious from reading a question whether the Survey writer wants a fact-based or opinion-based answer. With this in mind, avoiding this problem should be relatively simple. Here are a few guidelines to ensure that you are maintaining clear separation between fact and opinion and selecting the right type of question for the specific context:

• Ask yourself, “Is there a correct answer?” In other words, am I seeking specific, factual information? If the answer is “Yes,” we’re in the land of facts. If the answer is “No,” we’re dealing with opinion, estimation, or conjecture.
• If you are asking about something that happened in the past, you are probably seeking facts. For example, “How much revenue did your company book in Q1 2014?” is a factual question; the respondent is asked about something that has already happened and that can (theoretically) be looked up and verified.
• If you are asking about something that is currently taking place or might/will happen in the future, you are probably seeking an opinion or estimate. “How much revenue will your company book in 2014?” requires the respondent to estimate or guess about something that hasn’t happened yet.
• Factual questions can usually be clearly defined and often involve specific units. “For how many YEARS have you worked at your current job?” begs for a factual response.
• When you are seeking an opinion or estimate, SAY SO. Start your question with phrases like “In your opinion,” or “Please estimate…” to make it screamingly obvious to the respondent that you want them to speculate.
• Finally, follow factual questions with opinion questions, not the other way around. First ask “How tall is/was your mother?” and then ask “Please estimate how tall your son will be when he grows up.” The factual question sets the stage for the estimate, putting the respondent in the right frame of mind and giving context to their responses.