2 Technical notes

Selection of indicators

Indicators are selected based on three criteria:

  1. Availability of two or more data points for more than 50 per cent of the countries in the corresponding region or subregion;
  2. Ability to set a transparent target value;
  3. The metadata are clear and well-explained.

If any indicator fails to fulfil any of these criteria, it is excluded from the analysis.

Measures for tracking progress

Two principal measures are used to assess regional and subregional progress towards the SDGs: Current Status Index and Anticipated Progress Index. The indices answer two different questions:

  1. Current Status Index: How much progress has been made since 2000?
  2. Anticipated Progress Index: How likely will the targets be achieved by 2030?

The Anticipated Progress Index measures the gap between predicted value of the indicator and specified target value. Both indices are constructed at the level of sub-indicator (a series, disaggregation, or subcomponent of an indicator) and can be aggregated at indicator, target and goal levels as desirable.

In this analysis, the Current Status Index is presented at the goal level (snapshot) and Anticipated Progress Index at the target and indicator levels (dashboard and progress gap).

Current Status Index

Given a specified SDG target value for each indicator, the values for the current year and the year 2000 can be used to measure the progress made since 2000, in relation to the progress needed to reach the SDG target by 2030.

The Current Status Index is constructed in two steps:

  • Step 1 - A metric is developed for each indicator to measure the progress made (blue bar in Figure 1) which can be compared with the entire progress needed from 2000 to 2030.
  • Step 2 - To see how much progress has been made - and still needs to be made - to achieve the goal, the metrics computed in step 1 are combined into one index that indicates the "average progress made" and the "average progress required" on a fixed scale.

Denoting indicator values for 2000 and the current year by I0 and Icv and the target value for 2030 by TV, and setting the normalized values of the indicator at 2000 and 2030 at 0 and 10, respectively, the normalized value for the indicator at the current year on the scale of 0 to 10 can be calculated as:

\[I_{cv}^{N} = \frac{I_{cv}-I_{0}}{|TV-I_{0}|}\times D\]in which

\[D = \begin{cases} 10~~~~~~~~increasing~is~desirable\\-10~~~~~decreasing~is~desirable \end{cases}\] when a desirable direction is clear.

For parity indicators, the value is:

\[I_{cv}^{N} = 10-\frac{|TV-I_{cv}|}{|TV-I_{0}|}\times 10\] If the region (or subregion) has progressed since 2000, the average over all normalized values under each goal provides an index that is between 0 and 10. But if the region has regressed, the value is negative, indicating the size of regression.

If the current value for an indicator has already reached or exceeded the target value, the Current Status Index does not need to be calculated and is automatically is set to 10.

In an ideal situation, data would be available for all indicators associated with each goal and the Current Status Index would provide a robust measure comparable across all 17 goals. However regional data are available for less than 42 per cent of the defined SDG indicators, and coverage is uneven across the 17 goals. Since the assessment is sensitive to the addition of new indicators, the results must be interpreted with caution. The number of indicators and availability of data have substantially increased since the previous edition of this Report, thus the results of this analysis should not be compared with those of previous years.

Anticipated Progress Index

This index compares predicted (anticipated) progress with targeted progress. By predicting the indicator value for the target year and benchmarking the predicted value against the target value, the index provides a measure of how much progress towards the target will still be required by the end of the target year (2030), assuming the pace of progress is sustained. This can be interpreted as an anticipated gap in the target.

Denoting the predicted value of indicator I for the target year by It, and value in the base year by Ib, one can approximate the progress gap by P when no regression has occurred, and by 100 - P when the indicator value has regressed since the base year. If a desirable direction is clear from the target, the value of P is defined as:

\[P = \frac{|TV-I_{t}|}{|TV-I_{b}|}\times 100\] In the case of parity indicators, we consider no regression has occurred if \(|TV-I_{t}| \le |TV-I_{b}|\).

The Anticipated Progress Index is only calculated for indicators that are not expected to achieve the target. When the predicted value has already reached or exceeded the target or is expected to reach the target by 2030, the indicator is automatically classified as "will be achieved" and Anticipated Progress Index is set to 0.

Based on expected progress, the value of P ranges from 0 to 100. If there is a predicted regression from the current level, P will be greater than 100.

P may be interpreted as the extra effort or acceleration needed to meet the target when the value is less than or equal to 100, and 100 - P is size of regression when it is greater than 100.

Indicators are classified into three predefined achievement levels:

Disaggregated statistics

Disaggregation by sex, location or combination of age and sex was available for 24 indicators. To take disaggregated statistics into account, a disadvantaged group for each indicator was identified as the group that had worse value (depending on the desired direction) compared to the reference population. For instance, if the unemployment rate is 12 per cent among entire youth labour force population and this value is 10 per cent among youth males and 15 per cent among youth females, then the youth female group is considered disadvantaged. Progress on each indicator or series is then measured as average of progress on reference population and progress for disadvantaged group. By counting for disadvantaged groups, progress on each indicator is penalized for slow progress on one or more subpopulations. It ensures that the achievement of the target value is conditioned to the achievement of the least advantaged group to meet leave no one behind ambition of the SDGs.

In applying both measures of tracking progress, an acceptance threshold of minimum 2 per cent change was considered for progress/regression at indicator level. In other words, only if overall change over the period was more than a 2 per cent increase or decrease (depending on the actual and desired direction of change), the change was accepted.

Aggregation

In total, 134 indicators are used to compute the Current Status Index for SDG progress assessment in 2020. Of these, however, ten indicators did not provide sufficient data for 2030 predictions and were not used for Anticipated Progress index calculations. When more than one variation for an indicator exists (for example health worker density), or disaggregated statistics are available for an indicator or one of its series, all variants are used in calculations. Progress at each indicator is measured as average of progress over all variants of the indicator. Special care should be given to aggregating at all levels; series, indicator, target and goal. The objective is to give equal weights to all targets under each goal, and equal weights to all indicators under each target (see reference 4 for more details on how to weight series and indicators).

Extrapolation methods

Producing the two measures of progress requires prediction as well as imputation of missing values in the current and previous years. These values were estimated using a weighted regression model that uses time-related weights, assuming the importance attached to the indicator values should be proportional to how recent the data are.

Suppose that n data points are available on indicator I for a given region over a period of T years, and we are interested in estimating the indicator value for the year t.

\(T=t_{n}-t_{1}\) where \(t_{n}\) and \(t_{1}\) are the latest and the earliest years, respectively, for which data on indicator I are available. The time-related weights work as multipliers that inflate/deflate the rate of change in each period in proportion to its temporal distance to the target year (t). The time-related weight for the ith data point for a given country/region for estimating indicator value of the year t is:

\[w_{i} = \frac{|t-t_{1}|}{|t-t_{i}|}~~~(t_{1}<t_{i}<t_{n})\] Weights are then incorporated into a regression model used for different indicators. In a few exceptions where indicator is time-independent, time-related weights were not used (e.g., disaster-related indicators, ODA and other financial aids, etc.).

Setting regional target values

Of 169 SDG targets, only 30 per cent have specific (implicit or explicit) target values. For the rest, this report sets target values using a "champion area" approach. This is based on what has been feasible in the past and optimizes the use of available data. The idea is to identify the top performers in the region and set their average rate of change as the region's target rate. If we imagine all the top performers as belonging to one hypothetical area, this can be labelled as the region's champion area whose rate of change equals the average for the top performers for one specific indicator. This can then be considered the target rate for the region. In other words, if the region as a whole can perform as well as its champion area over the 15 years (SDGs era), we should expect to achieve the target value. Subsequently, the universal target value for the region can be derived by applying the rate of change in the champion area to the regional value in the base year. In this report, the regional value is the median value of the indicator over all countries for which data are available.

The main challenge with the champion area approach arises when dealing with two types of indicators:

  • Type i: Indicators for which there are insufficient data to estimate the rate of change at the country level
  • Type ii: Indicators for which most of the countries started from a very low level and made such rapid progress that the observed growth rate cannot reasonably be applied to the future. Examples of this include the proportion of parliamentary seats held by women, the proportion of marine areas protected and the percentage of the population using the Internet. These rapid changes may have been due to technological advances, exploitation of untapped resources, or a paradigm shift brought about by a development agenda such as Millennium Development Goals.

For these two types of indicators, an alternative approach is taken. Rather than using the rate of change, the top five performers are identified based on the latest available data. The region's target value for the champion area is then the average value for those five countries - using the largest or smallest values depending on whether the desirable direction of change is an increase or a decrease. Before identifying the top five performers, outliers were dropped to avoid bias.

Assume we set a target value for indicator I:

Case 1. At least two data points are available since 2000 for a number of countries that show a diverse range of changes. In this case, the earliest and latest available data for the five countries with the highest rates of change are used to calculate r, the average annual rate of change over the five highest rates of increase/decrease.

r is calculated in two steps. The first step is to estimate the geometric mean of average annual growth rate for each country based on the earliest and the latest indicator values. The second step is to take a geometric mean over the top five rates of change. It is often the case that one or few countries experienced exceptional growth. These outlier countries are dropped from calculations in order to ensure the average of the top five performers is a realistic and achievable, yet aspirational target for the rest of the countries.

Case 2. For indicators for which there are insufficient data to estimate country-level rates of change, the latest data for each country are used to calculate the target value:

Target value: Average over indicator values for the five countries with the largest or smallest values depending on whether the desirable change is an increase or a decrease, respectively (after dropping outliers as in case 1).

Finally, the target value for the indicator is calculated as:

When unavailable, the indicator value for the base year (\(I_{2015}\)) can be estimated by applying an appropriate extrapolation method (as described above).


Box - Confidence of results at the Goal level

Due to limitations on the availability of indicators, the results aggregated at the Goal level are based on a percentage of the total Global SDG indicators along with indicators from internationally recognized sources. While the latter are not intended to substitute the former, they shed light on targets where otherwise no analysis would have been possible. Therefore, they are taken into consideration when assessing the completeness of the evidence at the Goal level. The strength of the used evidence is thus defined as the following ratio:

\[ Evidence~Strength~Factor = \frac{T_{Used}+P_{Used}}{T_{Global}+P_{Used}} \] Where \(T_{Global}\), \(T_{Used}\), and \(P_{Used}\) represent, respectively, the total number of indicators in the Global SDG framework, the number of Global indicators used in the calculations, and the number of indicators from widely recognized international data sources used.

For ease of analysis, a strength symbol denotes the evidence strength factor according to the table below.