Methodology of the Focus 2030 - Women Deliver comparative study on gender equality across 17 countries on the eve of the Generation Equality Forum

The survey was carried out through an online poll in 17 countries (Australia, Argentina, Canada, China, Colombia, France, Germany, India, Japan, Kenya, Mexico, New Zealand, South Africa, Switzerland, Tunisia, the United Kingdom, and the United States) in primary local languages, including Arabic, Chinese, English, French, German, Hindi, Japanese, and Spanish.

Survey Questionnaire

Focus 2030, the Development Engagement Lab, and Women Deliver co-created the survey questionnaire, with a set of 23 questions based on the current literature and priorities of the global gender equality agenda. The survey was reviewed by UN Women. The full questionnaire can be found below.

The Polling Institute: Deltapoll

Deltapoll is an opinion poll institute based in the United Kingdom that also produces analysis and provides strategic advice. It was founded in 2018 in London by Martin Boon, Joe Twyman, and Paul Flatters. Deltapoll uses a panel of 750,000 adults in Great Britain and Northern Ireland, and global panels covering 60 million people across 45 countries worldwide, including the 17 countries in this particular study.

Setting up the Panel: Sampling

Panel respondents were recruited on the internet from a wide variety of sources including invitation via Internet service providers and recruitment through pop-up ads on websites. Each respondent receives a fixed bonus per month based on the number of surveys they have completed.
The 1,000 survey respondents in each of the 17 countries were sampled through quotas to ensure representation of the adult population of each country. In total, this comparative survey across 17 countries is based on a sample of 17,160 respondents.

Demographic Data Collected

Respondents were asked to self-identify their gender from three survey options: “male,” “female,” and “in another way.” “In another way” may include (but is not limited to): agender, genderqueer, non-binary, transgender female, transgender male, Hijra, and Two-Spirit, prefer not to say, gender identity not listed. The number of “in another way” self-identified respondents were very small in each country, and, hence, any related data analysis would not be statistically sound. In addition, information was collected regarding the respondent’s age (i.e., 18–24 years old, 25–44 years old, 45–60 years old, 60+ years old), education level (i.e., no formal education, some formal schooling but not university/college, university/college and beyond), income levels (categorized by quintiles), migrant status (i.e., refugee and/or asylum seeker, forcibly displaced within or from country of origin, economic migrant, none of the above), and race or ethnicity as appropriate for the country context and where applicable.

With regard to political orientation, respondents were to select an option on a 0 to 10 scale where 0 is left and 10 is right. In this report, a respondent is considered to have self-identified as we define “left-leaning” if they selected options from 0 to 3, “center” if they selected options from 4 to 6 and “right-leaning” if they selected options from 7 to 10.

For practical reasons, although in some countries a majority of people seem to self-identify as “center,” the analysis focuses on the contrast between “left-leaning” and “right-leaning” respondents in order to assess whether political orientation is associated with the opinions, knowledge, and experiences across issues.

Weighting and Data Analysis

For each country the raw data was weighted by gender, age, and region plus (where possible) past vote from the previous first-order election. The targets for these weights were derived from national census data along with official government statistics, large national surveys, and verified election data. The results tables were then produced in SPSS and formatted in Excel.

The data tables were then analyzed by Focus 2030, in partnership with The Development Engagement Lab (DEL) team and Women Deliver.

Survey Dates and Margin of Error

Responses were obtained online between July 24, 2020 and August 4, 2020 from all 17 countries. Based on a random sample of 1,000 respondents in each of the countries surveyed, the margin of error is +/- 3 percentage points with a 95% confidence interval.

Race/Ethnicity

The categories were taken, where possible, from the national census for each individual country. Where this was not possible, official government statistics and large-scale national surveys were used as proxies. Smaller sub-categories were then combined to create logical categories of sufficient size. For example, in Britain, “Black African”, “Black Caribbean,” and “Black Other” were combined into one single “Black” category.

Translation Methods

The translations of the questionnaires were conducted by a leading independent professional translation agency based in London who conducts translations for companies in the research and legal space.

Biases and Limitations

Representing the diversity of the world’s countries through a comparative survey of 17 countries was a challenge. Because such a project cannot, by definition, be exhaustive, choices had to be made in the selection of the countries to cover. Despite the inherent limitations of this unavoidably incomplete exercise, this comparative survey across 17 countries nevertheless provides a very significant portrait of the cause that concerns us worldwide: equality between women and men.

All surveys or polls are susceptible to methodological bias. Inevitably, the sample of respondents, while representative along some demographic indicators may not be an accurate representation of the adult population of each country in all its social or demographic parameters. External factors may affect the constitution of the panel: the willingness or personal interest to participate in the survey, the ease or difficulty of respondents to contribute through an online survey, geographical considerations allowing for more representation of urban areas in some countries, the specific context of COVID-19, and the unprecedented experience of lockdown, etc.

Thus, the fact that these surveys were conducted exclusively online did not allow the expression of people who would be deprived of any access to the internet. While the impact of an online survey is minimal in more developed countries, in countries such as Argentina, China, Colombia, India, Kenya, Mexico, South Africa, and Tunisia, it is inevitable that the respondents in the selected panel will be more representative of urban areas and more advantaged professional circles.

Furthermore, the specificities of each country surveyed in terms of social norms and attitudes had to be taken into account when structuring the samples and drafting the questionnaire. For example, questions on ethnicity were not possible in France, while questions on sexuality or sexual orientation were removed in Tunisia and adapted in Kenya, where homosexuality is either illegal or criminalized.
Given that China is a one state party and India might not have the same approach to what "left" and "right" is usually considered, voluntarily, this report does not rely on any political orientation from respondents in China and India.

Intersectionality

While all women face discrimination, the overlap between gender and other social identities can further disadvantage some women. Other grounds of discrimination, such as age, ethnicity, disability status, sexual orientation, religion, socio-economic origin, and migrant status, can in some cases exacerbate gender discrimination.

Acknowledging that the intersection between gender and other identities contributes to unique experiences of oppression and privilege, answers to the survey were analyzed according to respondents’ socio-economic characteristics, such as gender, age, income level, education level, and place of residence.

Given the limited sample sizes (1,000 respondents in each country) this report does not present the perceptions, attitudes, and experiences of people belonging to racial, ethnic, sexual, gender, and migrant minorities, It is nevertheless important to recall that they often experience more discrimination, while being excluded from decision-making processes.