URBAN WORK INDEX METHODOLOGY
The Building Blocks of the Urban Work Index
YouthfulCities has been ranking cities around the globe on youthfulness since 2013. Our understanding of what to measure is based on surveys with over 30,000 youth around the world to understand what they care about in their cities. Great cities to work in comes up as one of the main themes. For youth, work is not just about generating an income . It is a more comprehensive model that includes having an affordable city to live in, having the skills and education to increase their income over time, finding a first job that delivers some experience and then parlaying that experience into a more career-oriented future. For some it is also about starting a business. Where youth choose to live can impact affordability, eduction, employment and entrepreneurship dramatically. But how does the most mobile generation of youth ever choose a great city to work in? We have created the new Urban Work Index, ranking 21 cities across Canada. Measuring and analyzing what are the best cities for work, from the perspective of youth, is no easy task. In order to understand what elements are a priority for youth when considering their careers and futures, we consulted 170 youth across Canada about their opinions on the future of work in their cities. We developed 16 attributes of work stemming from our four themes: Affordability, Education, Employment and Entrepreneurship. For each of the attributes, we asked youth to tell us, on a scale of 1-10, how important the given attribute is for them. Then we finalized a list of 48 indicators to measure cities from work perspective for youth.
The Urban Work Index Methodology
One of our greatest challenges was developing a research methodology that provides results that are comparable across all cities.
Our methodology is made up of a few key steps:
1. Determining what to measure in cities In the past five years, using our quantitative Urban Attitudes Survey, we've asked over 30,000 youth in cities around the world what is most important to them. This led us to establish 20 Urban Attributes for our comprehensive index. For the present index, we've zeroed in on the work theme, and have asked over 170 Canadian youth to identify their priorities across 16 work-specific attributes specially designed for this report.
2. Recruiting a team of Urban Decoders All YouthfulCities data is collected by our teams of local youth researchers, who we call our “Urban Decoders’. They’re 15-29 years old, are connected, engaged, curious and inventive. They include students, entrepreneurs and employees of government and private enterprise. All of them exist as the driving force behind the youth-led regeneration in each of the cities we work in. We recruit, train, support and connect all of our Urban Decoders through the data collection process, and we continue to provide support with networking after project responsibilities are completed.
3. Collecting data to measure important attributes of cities Urban Work Index: A database of 21 Canadian cities that measures, compares and ranks the cities in our Index across 16 Urban Attributes using a total of 48 indicators. The indicators consist of primary and secondary data that Urban Decoders collect locally and submit using collaborative, cloud-based research workbooks.
4. Ranking the Index data: Once the data is collected, it is rigorously checked for accuracy and normalized. Once all data is normalized, the cities in the Urban Work Index are ranked using a scoring system that takes the normalized data from each of the 48 indicators and translates every data point into weighted scores. Overall scores are created from summing the average of the indicator scores across each Work Attribute.
IMPORTANT NOTE: The Survey data is not used as Index data. It is only used to build a more inclusive and credible weighting system for the Index. ie. Affordable Housing is more important to youth than Government Entrepreneurship Programs so the Index scores should reflect this.
Checking the primary and secondary YouthfulCities Urban Work Index data
Once work is collected and submitted all data goes through a rigorous internal review and check. Every data point is submitted along with a reliable source. The internal YouthfulCities team, along with external advisors, goes over the data points and sources, flags anything that does not hold up to scrutiny, and if necessary, uses our source database to replace the data point. All data checking is done collaboratively and can be tracked by all team members.
Normalizing the YouthfulCities Global Index data
Once data is checked it needs to be normalized in order to accurately compare each city:
Where necessary, data is converted to metric units
All cost-based data is normalized to CAD dollars
Since we’re motivated by a desire to measure cities from the perspective of youth, and since reliable average youth income data does not exist in the majority of the cities in our Index, we measure all cost indicators relative to one hour of minimum wage labour in CAD dollars in each city. For example:
If minimum wage in a city is $10 and the cost of a movie ticket is $12, the cost of a movie ticket, tied to minimum wage equals 1.2
Comparing data from different years
Wherever possible, data was gathered from the same calendar year. Given the different intervals of census deliveries and alternate data source availability, for some indicators we needed to compare data from different years. As a rule we only go back as far as five years.
Scale and boundary issues
For a very small number of indicators, city level data was unavailable. In these cases we collected data from province/state or national sources. To normalize the data collected from this larger sample we took the total population of a given city as a percentage of the total population of the broader area, and then multiplied the data collected from the broader area against the percentage that the city represented.
When data was not available for youth aged 15-29 we used a weighting system to allow us to use data from differently defined groups. For example, if population data was not available for youth aged 15 to 29 but was available for 15 to 24 years and 25 to 34 years, we used the 15 to 24 years data, and half of the total data from 25 to 34 years.
Hierarchy of data credibility
Primary and secondary data was collected from a number of sources. YouthfulCities collected primary data by talking to key sources in person, by email and by telephone. Secondary data collection was done largely through online research. Our sources include census reports, municipal offices and websites, non-governmental organizations, academic sources (e.g. journal articles, development indexes and reports) and other online sources (e.g. crowdsourcing sites like expatistan.com).
Ranking the YouthfulCities Global Index data
Once all data is normalized, the 21 cities in the Urban Work Index are ranked using a scoring system that takes the normalized data from each of the indicators and translates every data point into weighted scores via the following:
For each indicator we have decided if a high number or a low number is the most desirable for youth. For example:
In the number of entrepreneurship incubators indicator, a higher number of entrepreneurship incubators wins; versus,
In the youth unemployment indicator, a lower youth unemployment rate wins.
Raw scores are then translated into a value out of 100 using a relational scoring system:
For high number indicators the highest number gains 100 points for that city
For low number indicators the lowest number gains 100 points for that city
Points are then allocated to other cities based on a differential equation
100-(net difference to winning number x differential ratio)
Each city’s points are then translated into weighted scores based on the importance rankings determined for each urban attribute. Importance rankings come from a survey on Urban Work conducted with youth across Canada, which measured the average importance of the attributes for youth. Values listed are out of 10.
Cities' overall scores and rankings
The overall index scores result from summing the average of the indicator scores across each Urban Attribute. For example, the Transit Attribute is made up of 8 indicators. Each city gets a score for each indicator in the Transit Attribute. The average of these 8 indicator scores makes up the Attribute score. All 16 Attribute scores are summed in order to determine a city’s overall score and rank.
While reliable, robust and defensible data is available for the overwhelming majority of our indicators across all 21 cities, in an extreme minority of cases (<2%) reliable data simply does not exist. Where data does not exist we take a regional average in order to assign a score.
Obtaining reliable and comparable data for the 21 cities is the most difficult part of the YouthfulCities Urban Work Index process. We want to be as representative as possible in the data that we employ. If you feel you have a better way of measuring cities from a youth perspective, we want to hear from you!
YouthfulCities is a social enterprise, and therefore we will publicly release the scoring but not the individual data points. If there is interest in the data points please feel free to contact us. The data will be available for youth in order to encourage the use of the index for improving their cities. A small fee will be applied to this service for non-youth. A short explanation as to which data is requested and what use is intended will be part of the request process.