CHECK: Using Standardized Questions

Operators collecting quantitative data from stakeholder audiences for their public awareness program effectiveness must use a core set of survey questions (provided in Annex D of the Third Edition).

Quantitative data collection methods can include online, phone, mail, or face-to-face surveys or business reply card questionnaires that are often included with brochures or other printed materials.

Further, operators are required to use either the exact wording of the survey questions or the allowable edits and customization provided for each question. Customization can include specific language on pipeline type or product(s), for example, to make the question relevant to their pipeline system. Alternative questions can also be used to help determine if program objectives have been met. Deviations from the questions in Annex D or from the allowable edits and customizations should be documented in the operator’s effectiveness evaluation along with the reason for the deviation.

As noted in Annex D, customization is permitted as long as it does not change the intent of the question but allows an industry segment, coalition, or operator to benchmark performance and build comparisons over time. Although the pipeline industry has been collecting program data since 2007, the variety of data collection methods used have not—until now—been required to use the same core questions or collection methodology, which is critical for benchmarking and determining trends in the data.

Operators may supplement surveys by adding additional questions from Annex E or other questions not included in Annexes D or E.

Each question in Annexes D and E will provide the appropriate measurement actor (e.g., Reach, Recall, Message Understanding, or Behavior Intent), the question stem that should be used, the response categories that should be designed (e.g., yes/no/don’t know or a list of responses related to the question), whether it’s an aided or unaided question (indicating how the question should be formatted), and which program objective the question relates to (e.g., awareness, prevention, or recognition and response). Beneath each question are examples of the question with allowable edits or customizations.

For example, question D.1.3:

Question: From what you’ve read, seen, or heard, what are the kinds of things that might tell you that [a descriptor pipeline] is leaking? Please provide as many responses as come to mind.

Here are examples of allowable edits/customizations:

  1. From what you’ve read, seen, or heard, what are the kinds of things that might tell you that natural gas is leaking? Please provide as many responses as come to mind.
  2. From what you’ve read, seen, or heard, what are the kinds of things that might tell you that a crude oil pipeline is leaking? Please provide as many responses as come to mind.
  3. From what you’ve read, seen, or heard, what are the kinds of things that might tell you that an unodorized gas pipeline is leaking? Please provide as many responses as come to mind.
  4. From what you’ve read, seen, or heard, what are the kinds of things that might tell you that a gas utility line is leaking? Please provide as many responses as come to mind.
  5. From what you’ve read, seen, or heard, what are the kinds of things that might tell you that a pipeline is leaking? Please provide as many responses as come to mind.