Evaluation Metrics and Methodologies
Building a 10-Minute-City Model
Data collection and analysis of active modes use and infrastructure
Global protocol for community-scale greenhouse gas inventories
Greenhouse gas emissions
According to the European Environment Agency, greenhouse gas emissions (GHGs) are gases that ‘contribute to the natural greenhouse effect’. Six greenhouse gases ‘produced by human activities are: carbon dioxide, methane, nitrous oxide, hydrofluorocarbons, perfluorocarbons and sulphur hexafluoride.’ GHG emissions are estimated as CO2 equivalent emissions.
The total GHG emissions resulting from road transport depend on the quantity of travel and associated fuel consumption rates of vehicles, the type of fuel used by vehicles and energy sources, as well as the efficiency of vehicles and driving behaviour. Analyses can also include emissions from the production and disposal of vehicles and fuel. Assessments of GHG emissions at the city-level can consider different scopes of analysis: Scope 1 – emissions from transport inside a city boundary, Scope 2 – emissions electricity used for transport, Scope 3 – emissions from the portion of transboundary journeys occurring outside the city, and transmission and distribution losses from grid-supplied energy from electric vehicle use.
Congestion
While congestion is a main concern in many cities, measuring congestion levels in a homogenous way is challenging. Congestion encompasses different concepts, such as analysing travel along specific roads or city-level congestion. Estimating congestion along a road segment includes monitoring traffic flow or volume-to-capacity ratios. This can also compare peak hour travel speed to free-flow speed across specific routes or across a city network. Congestion can also be analysed from the point of view of different road users. From the perspective of public transport, commercial speed and punctuality or reliability indicators are often used in relation to congestion. Depending on the presence of separated public transport lanes or low emissions zones, and so on, general levels of congestion in a city may or may not reflect levels of congestion experienced by public transport users. Similarly for pedestrians and cyclists, travel volumes and speeds can also be measured, and congestion experienced by pedestrians and cyclists can be independent of congestion experienced by other road users.
Accidents and injuries
Improving road safety and reducing accidents and injuries is a primary goal for improving transportation. There is a significant amount of research into various ways to reduce road accidents as well as clear definitions related to this topic. Accidents refer to “an unwanted or unintended sudden event or a specific chain of such events which has harmful consequences.” A person injured is “any person who as result of an accident was not killed immediately or not dying within 30 days, but sustained an injury, normally needing medical treatment.” There are many related definitions of injury accidents, serious injuries, fatalities, etc. Many countries and cities regularly track and report road fatality rates at the country and city level. Analysis of accidents can also focus on ‘hot spots’ – locations where there are high accident rates –, or ‘near misses’ – locations where accidents are narrowly avoided (based on reports or observation).
Source: Glossary for transport statistics: Glossary_for_Transport_Statistics_EN_FINAL_WEB2_1.pdf (unece.org)
.
Modal share
Modal share is a main indicator for sustainable mobility which gives either the percentage of trips, person-kilometres, or vehicle-kilometres using each mode of transport. This generally refers to the urban or metropolitan level and is based on data from surveys, travel diaries, and traffic or passenger counts. While the general concept is clear, the indicator is complex since it often brings together multiple data sources and different ways to calculate or estimate each mode. Thus, modal split values reported by different sources for the same city within a similar time frame can vary more significantly than annual shifts in modal share. One important consideration is defining the boundary area that is being referred to as well as which people or trips within this boundary are counted, such as internal trips, cross-boundary trips, or resident trips. There are also challenges to obtaining accurate data for all of the travel modes. For example, travel surveys often focus on the main modes of transport for the longest portion of a trip rather than collecting information on each stage of a trip. This leads to undercounting of active travel modes. In addition, public transport routes may or may not align with the boundaries of an urban area.
Access to mobility services
Accessibility to mobility services has many different aspects which consider an individual’s or family’s ability to access public transport and modes of transport other than private automobiles. A key consideration is the availability and integration of various modes throughout a network, including having stops and stations within walking distance for the entire population as well as having sufficient frequency of services. It is also important to consider physical barriers to accessing the system for a variety of individuals, including vulnerable road users and people with disabilities. There are cultural aspects to consider related to safety and security of accessing and using various modes. Another key component is information availability about the services as well as technology and sources of used to communicate schedules and disruptions to users along with the financial cost of individuals or families to use the system. The term accessibility can also refer to a broader idea of access to all services in a city (such as the 10- or 15-minute city concept). This concept can be captured by looking at accessibility to primary services or the percentage of population living within close distance to primary services such as schools, doctor, pharmacy, grocery store, postal service and public meeting places. Due to the broad scope of accessibility, a variety of indicators or indices can be used for an analysis.
Air pollution
Air pollution has been regulated by air quality standards and vehicle emissions standards for decades as air quality has a significant impact on human health and quality of life. Standards related to reducing air pollutants have had significant results in improving air quality, however, progress is still needed to reduce harmful effects. At present in Europe, the main emissions measured which are associated with road transport are: nitrogen oxides (NOx), specifically nitrous oxide (N2O) , sulphur oxides (SOx), carbon monoxide (CO), methane (CH4) and non-methane volatile organic compounds (NMVOCs), particulate matter with diameters of 10µm/2.5µm or less (PM10/2.5), and ammonia (NH3) (European Environment Agency, 2023). The quantity of pollutants emitted depends on quantity of travel and associated fuel consumption rates of vehicles, the type of fuel used by vehicles and energy sources, as well as the efficiency of vehicles and driving behaviour.
Total quantity of air pollutants emitted by vehicles impact air quality along with other factors such as where pollutants are emitted, urban characteristics, and meteorological conditions (speed, wind direction, precipitation, etc.). These factors can lead to high concentrations of pollutants is certain cities or zones of a city. For understanding the contribution of a specific intervention towards reducing air pollution, it is important to measure or estimate the total quantity of air pollutant emissions. For understanding the connection of air pollutants to human health, air quality is an important indicator to consider along with exposure time to poor air quality.
Noise pollution
Noise pollution is caused by many sources including transportation. It has become a major health concern as it is linked to sleep disturbance, stress, and cardiovascular disease. Noise is measured in decibels, and for urban and transportation planning use, is generally reported as Leq equivalent. This describes the equivalent steady-state sound level, over a specific period of time, which contains the same acoustic energy as a varying sound level during the same time period. This information can also be expressed in a noise contour map depicting which locations experience certain levels of noise. Important considerations for understanding the impact of noise pollution are whether or not people are being exposed to a sound, for how long, during what time of day, what sound level.
User satisfaction
User satisfaction assessments can cover a wide range of topics and can be conducted by different agencies or institutions for different purposes. Some of the main topics are: Overall satisfaction within the PT service (rating scale e.g., 1 – 5-); reliability of service; accessibility, including cost and perceived cost; availability, including waiting time and travel time; perceived safety at stops and onboard -in city and in FUA, at day and night; comfort (cleanliness, availability of seats, crowding level..); ease of use, including ticketing and multimodality; staff courtesy; information provision (real-time information, route maps, timetables).
There are many methods cities can use to obtain input from citizens. This includes surveys, focus groups, serious games, expert interviews, in-depth interviews including with associations and activists, and key experts (professional drivers, local journalist, marketing & SUMP expert), complaints, online feedback, or FAQs, social media monitoring, the mystery shopper surveys (deploying trained assessors to anonymously evaluate the quality of service and user experience at various touchpoints within the PT system). For a more qualitative analysis of user satisfaction, protest actions, news articles, debates in policy arena, etc. can also be considered.