CONTENTS INTRODUCTION 4 1. ETHICAL CODES AND LEGAL REGULATIONS 2. RAISON D'ĘTRE OF THE EFAMRO QUALITY STANDARDS SECTION 1: MANAGING RESEARCH PROPOSALS GENERAL SALES CONDITIONS / COST PARAMETERS 3. RESEARCH PROPOSAL VS. PRICE QUOTE 4. CONFIDENTIALITY 5. PENETRATION ASSUMPTIONS 6. STIMULI AND TEST-PRODUCTS 7. LENGTH OF INTERVIEW 8. SUPPLEMENTARY ANALYSES 9. SUBCONTRACTING RESEARCH BACKGROUND 10. DECISION TO BE MADE / USE OF RESULTS 11. QUESTIONNAIRE/DISCUSSION GUIDE AREAS 12. DISCUSSION ON ALTERNATIVE METHODS METHOD 13. PRIMARY DATA COLLECTION METHOD 14. SECONDARY DATA 15. PILOT PHASE 16. PROPRIETARY VS. SYNDICATE SAMPLING 17. TARGETED SAMPLE SIZE 18. DEFINITION OF THE UNIVERSE 19. SAMPLING TECHNIQUE AND PROCEDURES 20. SELECTION OF INTERVIEWEES WITHIN HOUSEHOLDS FIELDWORK 21. BRIEFING METHOD 22. PLACE(S) OF INTERVIEW 23. FIELDWORK PLANNING 24. QUALITY CONTROL PLANNED DATA PROCESSING & REPORTING 25. OPEN-ENDED QUESTIONS 26. WEIGHTING OF THE DATA 27. REPORTING FORMAT SECTION 2: CONDUCTING RESEARCH PROJECTS PROJECT SCHEDULE (AFTER COMMISSIONNING) 28. PLANNING THE RESEARCH 29. TIMETABLE OF THE FIELDWORK DATA COLLECTING & PROCESSING 30. RECORDS OF ALL FIELDWORK OPERATIONS 31. INFORMATION RECORDS ABOUT INTERVIEWERS 32. DATA PROTECTION 33. CLIENT'S ANONIMITY 34. RESPONDENTS CARD 35. DATE & LENGTH OF INTERVIEW 36. INTERVIEWER WORKLOAD 37. FIELDWORK MONITORING & CONTROL 38. ESTABLISHING CODING FRAMES 39. CHECKING CODES AND ENTRIES 40. DATA EDITING, OR "CLEANING" 41. DATA CLEANING FILE THE RESEARCH REPORT 42. COMPLETENESS FOR REPRODUCIBILITY 43. SEPARATION OF INTERPRETATION FROM RESULTS 44. VALIDITY OF THE RESULTS 45. DEFINITION OF ALL VARIABLES 46. DATA IN ELECTRONIC FORMAT MANAGEMENT OF SUB-CONTRACTORS 47. NECESSARY TRANSPARENCY 48. AGENCYS FULL RESPONSIBILITY 49. OTHER RESEARCH AGENCIES SECTION 3: SPECIFIC TYPES OF MARKET RESEARCH MULTINATIONAL RESEARCH 50. CO-ORDINATION OF MULTINATIONAL PROJECTS 51. TRANSLATIONS QUALITATIVE RESEARCH 52. MODERATORS/INTERVIEWERS STUDY PREPARATION 53. CLIENT OBSERVATION OF GROUP DISCUSSIONS & INTERVIEWS 18 54. RECORDING 18 55. QUALIFICATIONS OF GROUP MODERATORS AND INTERVIEWERS 18 56. USE OF COLLABORATORS/FREELANCERS 18
INTRODUCTION 1. ETHICAL CODES AND LEGAL REGULATIONS: All research projects will be executed in full accordance with the ICC/ESOMAR International Code of Marketing and Social Research Practice and its accompanying notes and, optionally, the relevant specific ESOMAR Guidelines (listed in Appendix) and in accordance with National Market Research Professional Codes and Guidelines where these operate. Projects will also comply with all legislation on data protection and on fair competition and, within the European Union, with the following E.U. Directives:
(as translated into National laws in member states of the European Union) 2. Raison d'ętre of these EFAMRO quality Standards: Most National Market Research Organisations associated with EFAMRO have encouraged their members to set up a Quality Assurance management, e.g. the ISO 9001 / 9002 certification. Some countries also have a Personnel Qualification system in place. A majority of EFAMRO members have issued Quality Standards documents (for example: "MRQSA", in the UK, which has now become a "British Standard"), in order to define a minimum mandatory level of quality, to be met at all times, in any research projects carried out by the Agencies members of the National Associations. The growth of International Research, the continuous concentration of the Market Research Industry, as well as the necessity to act together in Europe to ensure that development of data protection legislation takes account of the special needs of market research, have emphasised the need to develop a European set of Quality Standards. These new constraints have been defined collectively and are freely accepted by EFAMRO Member Associations, as an Industry self-imposed discipline. The EMRQS covers different areas from the ICC/ESOMAR Code although with some overlap. Consistent with the ISO 9000 Quality Assurance organisation, the EFAMRO Quality Standards aim at raising further the minimum quality that Clients may expect from any Research Agency in Europe, bringing the Industry a further step towards the "Single European Market".
Section 1: MANAGing RESEARCH PROposals General Sales Conditions / Cost Parameters: 3. RESEARCH PROPOSAL VS. PRICE QUOTE: A Price quote relates simply to the costs of implementing a research project to a predetermined methodology. It includes:
A Research Proposal differs from a Price Quote in that, in addition to the items listed above it will discuss the brief from the Client and propose the most appropriate solution to the research issues raised. 4. CONFIDENTIALITY: The strict confidentiality of all component parts of a research proposal is an ethical requirement, which binds both sides: the Research Agency as well as the Client. On one hand, the details of the Client's requirement will be treated in strict confidentiality by all tendering Agencies. On the other hand, unless otherwise agreed, the copyright of the Research Proposal and all other research documents (such as a questionnaire, discussion guide, etc.) remain the property of the Research Agency and no detail of any offer will be divulged to any third party. 5. PENETRATION ASSUMPTIONS: A proposal shall always indicate, when and if possible, what were the penetration assumptions (and their sources) used in preparing the cost and timing of the research. A proposal shall always state (or produce the best available estimate of) the proportion of the total population that the sampling universe represents. Any significant change observed during the early stages of a project from the original assumptions shall be brought to Client's attention. Any implications on the cost or timing of the project will be discussed and agreed with the Client. 6. STIMULI AND TEST-PRODUCTS: The proposal shall describe precisely which sort of stimulus materials and test-products if any will be needed and in what quantities. The proposal shall state whether any provision for these materials and products have been included in the proposed budget or not. It shall also state whether the products have to be returned to the client after the project has been completed. The proposal shall also state that any product and/or stimuli supplied by the client must comply with all legal requirements. The client is responsible for any claims or legal actions that may result from the use in the research of any products and/or stimuli supplied by the client and must release the agency from all consequent liability. 7. LENGTH OF INTERVIEW: A proposal shall always state the length of interview (either in terms of time, or question units), or of the group discussions, on which the cost had been established. Should the final questionnaire / discussion guide required by the Client exceed this length, then the cost and possibly the timing of the project may be subject to revision.\ 8. SUPPLEMENTARY ANALYSES: A proposal shall contain an indication of how far the data will be analysed, or over what period of time any supplementary analysis can be provided as a normal part of the agreed contract. Beyond this limit, any supplementary analysis will be invoiced, as an additional service not included in the main contract. 9. SUBCONTRACTING: The proposal shall clearly indicate where sub-contracting of any significant part of the research project is intended.
Research Background: 10. DECISION TO BE MADE / USE OF RESULTS: A proposal shall outline the research objectives and, where appropriate, indicate what marketing or other decisions may be affected by the results of the study. 11. QUESTIONNAIRE / DISCUSSION guide TOPIC AREAS: The proposal shall outline the topic areas to be covered by the questionnaire / discussion guide or other data collection tools. The precise formulation of the questionnaire / discussion guide will be determined only after the Client has formally commissioned the project. 12. DISCUSSION OF ALTERNATIVE METHODS: When several types of data, or several alternative data collection methods might be relevant to the problem, the proposal should discuss the advantages and disadvantages of the alternative methods that could be used. The reasons for choosing one particular approach rather than another will be indicated. When particular research techniques are to be used (such as projective or similar techniques in qualitative studies or statistical techniques such as conjoint analysis in quantitative studies), their general characteristics must be specified, along with the reason why they are the best response to the briefing. The choice of quantitative or qualitative methodology must also be justified when relevant. Method: 13. PRIMARY DATA COLLECTION METHOD: The proposal shall contain a precise description of the recommended data collection method. Any stimulus-material needed shall be thoroughly explained, in terms of purposes and format. If the approach is experimental, or developed specifically for the project, (as opposed to be a well-known, established data collection approach) this also shall be clearly stated. 14. SECONDARY DATA: If the method proposed includes any secondary data to be collected or used, the proposal shall outline:
Any known limitations to the secondary data to be used should be specified. 15. PILOT PHASE: The proposal shall describe the objectives and method of the pilot phase of the research, if any. 16. PROPRIETARY VS. SYNDICATE: The proposal shall always state clearly whether the data collected will be proprietary (exclusive) or syndicated (shared), or which parts of the results will respectively be proprietary or syndicated.
Sampling: 17. targeted SAMPLE SIZE: A proposal shall always state the anticipated size of the effective sample to be interviewed. In case of a multi-stage research, the minimum sample size, or the final sample size shall be specified. As regards group discussions and individual depth interviews, the number of individuals to be interviewed and/or the number of groups and intended number of participants in each shall be specified. 18. DEFINITION OF THE UNIVERSE: A proposal shall define exactly who in the population is eligible for interview, with a precise description of the criteria and procedure to be used for the selection of individual respondents. 19. SAMPLING TECHNIQUE AND PROCEDURES: A proposal shall describe which sampling approach has been chosen as well as the technique and procedures: random, quasi-random, or quotas. The anticipated number of sampling points, the place and description of the interviewing venues, the anticipated number of interviewers, or moderators (for group discussions) and any other important parameters impacting on cost and quality, will be indicated in the proposal, as well as any incentive for respondents to participate in the study. Any pre-determined criteria to be used when selecting (or excluding ) respondents from interview (such as users of a particular product; whether or not the respondent is employed by an advertising agency; etc) must also be specified. 20. SELECTION OF INTERVIEWeeS WITHIN HOUSEHOLDS OR ORGANISATIONS: When one person only is to be selected within a household or an organisation, the proposal shall describe the procedure that will be used to make this choice. In qualitative studies, the minimum length of time that must have elapsed since participants last took part, if ever, in a group discussion or individual depth interview will be indicated. Proposals for such studies must also make clear how respondents will be recruited (afresh, from a pre-recruited panel or selected by some other means). Fieldwork: 21. BRIEFING METHOD: The Research Proposal shall contain a description of the briefing method to be used with supervisors, interviewers and recruiters,, e.g.: face-to-face meeting with all interviewers, with supervisors only, or just mailed instruction to everyone, e-mail, telephone, etc. 22. PLACE(S) OF INTERVIEW: The Research Proposal shall state where the interviews will take place: in the street, at home, at the place of work, in a hall, a mall, etc. If respondents have been pre-recruited for interview, or if a study is conducted in several phases, there may be several places of interviews. In such cases, all circumstances of all types of interviews will be outlined. If the proposed location in which interviews are to be conducted requires prior permission to be obtained (such as in a shop) this requirement should be made clear in the proposal. As well as location, the day of week and time of day when interviews will be conducted should also be specified. In qualitative studies, the location of interview and/or group discussions must be specified (in home; in a hotel; at an office) and whether or not viewing facilities are available and/or whether the client may attend any of the groups or individual interviews 23. FIELDWORK PLANNING: If there is a deadline for the availability of the results, or the fieldwork or recruitment of respondents for interview or participation in group discussions must absolutely be carried out before a certain date or on certain specific days or at specific times of day, then the proposal should state the latest date by which the agreement has to be received from the Client in order to meet the time requirements. The Client must also be informed of the latest date for the receipt of products or stimuli to be used during fieldwork. 24. QUALITY CONTROL: If and when requested by the Client, the Agency shall supply information about:
Planned Data Processing & Reporting: 25. OPEN-ENDED QUESTIONS: The anticipated number of open-ended questions shall be stated in the proposal, as well as the method planned for the preparation of the coding frames. The proposal will indicate also whether verbatim quotes will be supplied, as well as or instead of frequency tables. 26. WEIGHTING OF THE DATA: The nature of any anticipated weighting of the data to be applied shall be described in the proposal. 27. REPORTING FORMAT: The proposal shall state the format of the results to be delivered at the end of the research: full report, summary report, aggregated tables, or individual data files. The documents may also be supplied on paper, or in electronic format, or in both ways: this shall be specified in the proposal, as well as the number of copies of the report included in the cost quotation. In qualitative studies the proposal shall state whether verbatim comments from interviews and/or group discussions will be included in the report The proposal shall also specify whether a presentation of the results is included or not in the budget, or proposed as an option.
SECTION 2: Conducting research projects
Project Schedule (After Commissioning): 28. PLANNING THE research: Once a project has been commissioned, it is the first task of the Agency to issue a planning document showing the timetable for all important stages of the study, where relevant:
A key Agency contact will be identified and the Client shall be advised promptly in case of any change to the schedule, including recommendations or suggestions for action to be taken, in the event of unforeseen circumstances arising. 29. TIMETABLE OF THE FIELDWORK: Where requested by the Client, the Agency shall issue a complete timetable (dates, times and locations) of all fieldwork operations, including the briefing(s) of interviewers, recruiters or moderators. The Agency shall facilitate the Client's attendance at some stages of the fieldwork, when requested, provided that this:
Data Collecting & Processing: 30. RECORDS OF ALL FIELDWORK OPERATIONS: In order to make available all elements necessary for analysing all discrepancies between the intended and the obtained structure of the sample, the Agency shall maintain records of:
To fully clarify the differences between the intended and the obtained samples, the following elements will be recorded and analysed, where relevant:
In the case of qualitative studies, records will be kept of each respondent recruited, the briefing of interviewers/moderators (unless no separate briefing is required because the interviewers/moderators working on the study were involved in the original briefing by the client) and the number of participants actually participating in each group discussion.
31. INFORMATION RECORDS ABOUT INTERVIEWERS: For each field-based interviewer, recruiter, auditor, or telephone-centre interviewer or recruiter, a file shall be maintained containing the following documented information:
Types of projects worked on such as product tests, business to business studies, group recruitment, structured quantitative surveys, etc
32. DATA PROTECTION: All fieldwork and data recording operations will be carried out in accordance with any relevant data protection legislation. Neither a questionnaire, audio or video tape where a respondent may be identifiable, nor any other record of data bearing the name of a respondent will be given to the Client or any third party, unless explicit consent of the respondent has been received and strictly for research purpose. No record containing identifiable personal details of the respondents shall be kept by the Agency for a longer time than necessary for the control of fieldwork and subsequent quality control checks and possible re-interviewing. When respondent or participant recruitment involves the use of lists supplied by the client, the use of such lists must comply with data protection legislation. 33. CLIENT ANONYMITY: The Client's Identity may not be revealed to respondents, except when the Research Agency and the Client jointly decide there are valid reasons to do so. 34. RESPONDENTS CARD: At the end of any face to face interview or group discussion, every respondent will be given a document with a note of thanks for his (her) co-operation as well as the name and address of the research Agency, with a telephone number. When conducting interviews by telephone a call back telephone number shall be offered at the end of the interview. 35. DATE & LENGTH OF INTERVIEW: The date and duration of all interviews will be recorded. In the case of interviews or discussion groups that are audio or video tape recorded, the tape will be labelled with the date of the recording. The duration will be clear from the recording. 36. INTERVIEWER WORKLOAD: The fieldwork records shall enable the number of questionnaires completed per interviewer to be analysed. 37. FIELDWORK MONITORING & CONTROL: The methods and proportion of interviews to be monitored and controlled (including respondents recruited for group discussions) will be specified in the proposal and confirmed in the report. Each Agency is free to establish its own standards of monitoring and control, with a specified minimum proportion of call-backs, listening, or monitoring, to be carried out amongst the interviews. The fieldwork supervisors will analyse, for each interviewer, the times of the day when the interviews were made, the frequency per day and the "strike rate" (proportion of successful interviews per 10 calls), where relevant. In qualitative studies, the Agency shall describe the system used to monitor and control the recruitment of group discussion participants or interviewees. 38. establishing coding frames: The Research Agency shall ensure that coders working on a project are briefed, or provided with briefing instructions, to include the following:
The Research Agency's normal practice in respect of code frames for open-ended questions (for example the proportion or minimum absolute number of interviews and the method used) shall be documented to the Client, on request. Within the code frames, the "No Answers" will always be distinguished from the "Dont know" answers. And the catch-all "Others" category shall not normally exceed 10% of the respondents and must never lead to any distortions in the distribution of responses. 39. Checking codes and entries:
In addition, the minimum percentage checked shall be specified, as appropriate, according to the type of research. 40. DATA EDITING, OR "cleaning": If and when requested by the Client, the Agency shall document the method of editing /data checking used after entry (report /or forced editing). Report Editing is checking that leads back to the questionnaires. When report editing is used, there shall be a record of decisions taken as a result of the editing reports. Forced Editing is automatic checking performed by a consistency verification and correcting program. When forced editing (often called "data cleaning") is used, the logic of the forcing shall be written down, submitted to the Client's approval if he wishes so, and test runs carried out, with the results documented, to ensure that forcing has the desired effects. For continuous research, where there is the need to handle failures without any delay, the rules developed to this purpose will be documented and applied immediately (so that decisions made need not be reviewed or confirmed on each occasion). In all cases, the main procedures and rules of data cleaning shall be documented in a Technical Appendix of the Research Report or made available on request. 41. DATA CLEANING FILE: Care shall be taken by the Agency in managing project files when data cleaning is conducted to ensure that:
When requested, the Client will be given access to such files for his project. The Research Report 42. COMPLETENESS FOR REPRODUCIBILITY: The Research Report shall always be sufficiently detailed to allow any reader to replicate the study or perform full secondary analysis. The description of what has been studied, what has been found, shall be thoroughly documented. All method parameters, data collection techniques, stimulus materials used and analysis tools applied to the data shall be described in detail. In particular, the research report shall include details of the following whenever relevant:
As the Agency will deem appropriate and depending of the research background and type of study, these technical details will be placed: In the Management Summary,
The following elements will also be included where relevant and must always be available on request:
43. SEPARATION OF INTERPRETATION FROM RESULTS: In both quantitative and qualitative survey reports the interpretation of the results (implications and recommendations) shall be clearly separated from the description of the data itself, when presenting as well as when reporting the study.
44. VALIDITY OF THE RESULTS: For many possible reasons, the results of any market research project may be affected by a certain degree of uncertainty. It is an ethical obligation for the Agency to mention explicitly in the research report all identified sources of potential inaccuracy in the results. For example, it will be essential to determine whether any apparently significant results may have been caused by a technical artefact, rather than by genuine causes. For example, the sampling method may have induced a "design effect". Or, an exogenous event in the course of the study may have occurred, which may have severely influenced the meaning of the results. On such occasions, the Agency shall identify and review all the implications of such "noises" on the research results. 45. Definition of all Variables: All variables used in the analysis of the data will be clearly and completely defined. Any indexing, scoring, scaling or clustering shall be fully explained with a table showing all parameters of the groupings of respondents. The Client is always entitled to see the tabulation of every single individual question asked. Each of the component parts of any combination variable shall be clearly identified so as to make the logic fully comprehensible to every reader. These technical details will appear:
The only exception to the above arises when a research analysis, model, or simulation exercise is proprietary and therefore confidential. 46. Data in Electronic Format: When data is to be made available in an electronic form, the precise format used shall be agreed in advance, between the Client and the Agency. Management of Sub-Contractors 47. Necessary transparency: Any anticipated sub-contracting of any substantial part of a research project shall be explicitly mentioned in the research proposal. Any unforeseen change, during the subsequent course of the project shall be brought to Clients attention. It is the Agency's responsibility to ensure that sub-contractors and freelancers work to the same level of Quality Standards as themselves.
48. AGENCYS FULL RESPONSIBILITY: The agency contracted by a client to carry out a project remains entirely responsible for all work carried out in connection with the project, including any important part of the work which may be sub-contracted 49. OTHER RESEARCH AGENCIES: The criteria for selection of other research agencies for sub-contracting will be:
Membership of a National Market Research Association or ESOMAR and listing in the ESOMAR Directory of Agencies, shall be sought after, because this implies the adherence to a National Code of Conduct or the ICC/ESOMAR International Code of Marketing and Social Research Practice, as well as to the Guidelines enforced by these professional bodies. Although not mandatory, ISO 9000 certification is recommended. An Agency working to the EFAMRO Quality Standards and/or National Quality Standards shall sub-contract to other Agencies only if they can offer the same level of quality.
SECTION 3: Specific types of market research MULTINATIONAL RESEARCH: 50. co-ordination of multinational projects: An Agency in charge of co-ordinating an international research project shall be bound by the same obligations of transparency and accountability as for a one-country project. In the case of international, multi-country research, the planning of the project (see Art. 28 & 29 above), the sub-contracting transparency (Art.47) and the Agency responsibility (Art 48) become even more crucial to the overall quality provided to the Client. The Client of an international project shall be informed about:
All these issues will be clarified in the proposal or in the planning document, issued by the co-ordinating Agency. 51. TRANSLATIONS: The Agency shall specify how the questionnaires, discussion guides, and other sensitive documents will be translated and adapted when and if needed, as well as how the translations will be checked. All translators will be nationals, mother-tongue speakers of at least one of the languages concerned. If they are mother tongue speakers of only one of the languages, they should also have been fluent for some years in the other languages concerned. QUALITATIVE RESEARCH: 52. MODERATORS / INTERVIEWERS STUDY PREPARATION Each moderator will familiarise himself with the objectives of the research study, the issues to be explored, the stimulus material and the particular techniques to be used, as relevant, prior to the commencement of fieldwork. This requirement is subject to the availability of stimuli material, or any other material to be used, whose availability is beyond the control of the agency. 53. CLIENT OBSERVATION OF GROUP DISCUSSIONS & INTERVIEWS All agencies will ensure that recording and observation of fieldwork for qualitative studies conforms to the requirements of the ESOMAR Guideline on Tape and Video Recording and Client Observation of Interviews and Group Discussions. The client or his representatives may view interviews or group discussions, via either a one-way mirror or CCTV, but participants must be informed of and agree to this at the start of the session. It shall be deemed sufficient proof that this has been done if a statement covering this issue is included and recorded at the beginning of each group discussion or interview. If it is possible that the client may be able to identify any of the participants, either as a personal acquaintance or as a member of an organisation, the client must not be allowed to view the session. All Clients viewing individual interviews or group discussions must sign a document stating that any information they obtain while observing interviews or group discussions will be used solely in relation to the research project concerned. They must also agree to be bound by the ICC/ESOMAR International Code of Marketing and Social Research Practice as well as National Professional Codes and Guidelines and must be offered copies of these if they are not already available to them. 54. RECORDING The instruction in Rule 7 of the ICC / ESOMAR International Code of Marketing and Social Research Practice, regarding the obligation to inform respondents that they are being recorded, will always be observed, as will points 1 to 8 of the ESOMAR Guideline: "Tape and Video-Recording and Client Observation of Interviews and Group Discussions" and the relevant regulations of National Professional Codes and Guidelines. 55. QUALIFICATIONS OF GROUP MODERATORS AND INTERVIEWERS If requested by the Client, the research agency will specify the experience and qualifications of discussion group moderators and interviewers for depth interviews. 56. USE OF COLLABORATORS, FREELANCE INTERVIEWERS AND INDEPENDENT GROUP MODERATORS Where independent group moderators, collaborators or freelancers are used (i.e. individuals who are not on the payroll of the research agency) in connection with a project, the agency shall include a confidentiality clause in the contract with such individuals relating to any information they may obtain during the conduct of the project. The Client shall receive on request a full description of the qualifications and experience of all individual collaborators, freelancers or group moderators working on the project concerned. |