2019 Canadian Index Methodology
The Building Blocks of the Urban Work Index
The Urban Work Index Methodology
One of our greatest challenges was developing a research methodology that provides results that are comparable across all cities.
Our methodology is made up of a few key steps:
1. Determining what to measure in cities.
In the past five years, using our quantitative Urban
Attitudes Survey, we've asked more than 30,000 youth in cities around the world what is most important to them.
This led us to establish the 21 Urban Attributes that make up our Index.
2. Recruiting a team of Urban Decoders.
All YouthfulCities data is collected by our team of local youth researchers, who we call our Urban Decoders. They’re 15-29 years old, connected, engaged, curious and inventive. They include students, entrepreneurs and employees of government and private enterprise. All of them exist as the driving force behind the youth-led regeneration in each of the cities we work in. We recruit, train, support and connect all of our Urban Decoders through the data collection process, and we continue to provide support with networking after project responsibilities are completed.
3. Collecting data to measure important attributes of cities.
The 2019 YouthfulCities Canadian Index is a database of 22 Canadian cities that measures, compares and ranks the cities across 21 Urban Attributes using a total of 123 indicators. The indicators consist of primary and secondary data that Urban Decoders collect locally and submit using collaborative, cloudbased research workbooks.
4. Ranking the Index data
Once the data is collected, it is rigorously checked for accuracy and normalized. The cities in the 2019 Canadian Index are ranked using a scoring system that takes the normalized data from each of the 123 indicators and translates every data point into weighted scores. Overall scores are created from summing the average of the indicator scores across each attribute.
IMPORTANT NOTE: The Survey data is not used as Index data. It is only used to build a more inclusive and credible weighting system for the Index. ie. Affordable Housing is more important to youth than Government Entrepreneurship Programs so the Index scores should reflect this.
Checking the primary and secondary YouthfulCities Urban Work Index data
Once work is collected and submitted all data goes through a rigorous internal review and check. Every data point is submitted along with a reliable source. The internal YouthfulCities team, along with external advisors, goes over the data points and sources, flags anything that does not hold up to scrutiny, and if necessary, uses our source database to replace the data point. All data checking is done collaboratively and can be tracked by all team members.
Normalizing the YouthfulCities Global Index data
Once data is checked it needs to be normalized in order to accurately compare each city:
Where necessary, data is converted to metric units
All cost-based data is normalized to CAD dollars
Since we’re motivated by a desire to measure cities from the perspective of youth, and since reliable average youth income data does not exist in the majority of the cities in our Index, we measure all cost indicators relative to one hour of minimum wage labour in CAD dollars in each city. For example:
If minimum wage in a city is $10 and the cost of a movie ticket is $12, the cost of a movie ticket, tied to minimum wage equals 1.2
Comparing data from different years
Wherever possible, data was gathered from the same calendar year. Given the different intervals of census deliveries and alternate data source availability, for some indicators, we needed to compare data from different years. As a rule we only go back as far as five years.
Scale and boundary issues
For a very small number of indicators, city level data was unavailable. In these cases we collected data from province/state or national sources. To normalize the data collected from this larger sample we took the total population of a given city as a percentage of the total population of the broader area, and then multiplied the data collected from the broader area against the percentage that the city represented.
When data was not available for youth aged 15-29 we used a weighting system to allow us to use data from differently defined groups. For example, if population data was not available for youth aged 15 to 29 but was available for 15 to 24 years and 25 to 34 years, we used the 15 to 24 years data, and half of the total data from 25 to 34 years.
Hierarchy of data credibility
Primary and secondary data was collected from a number of sources. YouthfulCities collected primary data by talking to key sources in person, by email and by telephone. Secondary data collection was done largely through online research. Our sources include census reports, municipal offices and websites, non-governmental organizations, academic sources (e.g. journal articles, development indexes and reports) and other online sources (e.g. crowdsourcing sites like expatistan.com).
Ranking the YouthfulCities Global Index data
Once all data is normalized, the 21 cities in the Urban Work Index are ranked using a scoring system that takes the normalized data from each of the indicators and translates every data point into weighted scores via the following:
For each indicator we have decided if a high number or a low number is the most desirable for youth. For example:
In the number of entrepreneurship incubators indicator, a higher number of entrepreneurship incubators wins; versus,
In the youth unemployment indicator, a lower youth unemployment rate wins.
Raw scores are then translated into a value out of 100 using a relational scoring system:
For high number indicators the highest number gains 100 points for that city
For low number indicators the lowest number gains 100 points for that city
Points are then allocated to other cities based on a differential equation
100-(net difference to winning number x differential ratio)
Each city’s points are then translated into weighted scores based on the importance rankings determined for each urban attribute. Importance rankings come from a survey on Urban Work conducted with youth across Canada, which measured the average importance of the attributes for youth. Values listed are out of 10.
Cities' overall scores and rankings
The overall index scores result from summing the average of the indicator scores across each Urban Attribute. For example, the Transit Attribute is made up of 8 indicators. Each city gets a score for each indicator in the Transit Attribute. The average of these 8 indicator scores makes up the Attribute score. All 16 Attribute scores are summed in order to determine a city’s overall score and rank.
While reliable, robust and defensible data is available for the overwhelming majority of our indicators across all 21 cities, in an extreme minority of cases (<2%) reliable data simply does not exist. Where data does not exist we take a regional average in order to assign a score.
Obtaining reliable and comparable data for the 22 cities is the most difficult part of the YouthfulCities Urban Work Index process. We want to be as representative as possible in the data that we employ. If you feel you have a better way of measuring cities from a youth perspective, we want to hear from you!
YouthfulCities is a social enterprise, and therefore we will publicly release the scoring but not the individual data points. If there is interest in the data points please feel free to contact us. The data will be available for youth in order to encourage the use of the index for improving their cities. A small fee will be applied to this service for non-youth. A short explanation as to which data is requested and what use is intended will be part of the request process.
Understanding what youth want in their city is no easy task. Since 2013, we have asked over 30,000 youth around the world to tell us what they think makes a city youthful. This has resulted in the creation of our Live, Work and Play wheel, which holistically captures all of the elements of a city that youth care about.
We asked youth to tell us how important each attribute on the wheel is, on a scale of 1-10. For the 2019 Index, YouthfulCities surveyed over 300 youth across Canada, and asked them to rank these elements in their city. From there, we calculate the average level of importance for each attribute. For each attribute, we collect data on various indicators that holistically make up that attribute and can be measured empirically.