Progress Assessment Methodology
Asia and the Pacific SDG Progress assessment is based on the global indicator framework for the 2030 Agenda for Sustainable Development as adopted by the General Assembly on 6 July 2017. Data used in this analysis are sourced from the Global SDG Indicators Database maintained by Statistics Division of the United Nations Department of Economic and Social Affairs. When sufficient data on a defined SDG indicator are not available, additional indicators from internationally recognized sources were used. The indicators are available on the ESCAP SDG Gateway Data Explorer1 along with the information on country groupings and definitions.2 Average values of indicators at the regional and subregional levels are used instead of weighted aggregates to avoid bias towards bigger countries or economies.
Selection of indicators
Indicators are selected based on two criteria:
- Availability of two or more data points for more than 50 per cent of the countries in the corresponding region or subregion;
- Ability to set a quantitative target value.
If any indicator fails to fulfil any of these criteria, it is excluded from the analysis.
The list of indicators with respective target values is published in the Asia and the Pacific SDG Progress Report.
Measures for tracking progress
Two principal measures are used to assess regional and subregional progress towards the SDGs: Current Status Index and Anticipated Progress Index. The indices answer two different questions:
- Current Status Index: How much progress has been made since 2015?
- Anticipated Progress Index: How likely will the targets be achieved by 2030?
The Anticipated Progress Index measures the gap between predicted value of the indicator and the specified target value. Both indices are constructed at the level of sub-indicator (a series, disaggregation, or subcomponent of an indicator) and can be aggregated at indicator, target and goal levels as desirable. In this analysis, the Current Status Index is presented at the goal level (snapshot) and Anticipated Progress Index at the target and indicator levels (dashboard and indicator progress).
In an ideal situation, the Current Status Index would provide a robust measure comparable across all 17 goals. However, given data availability is limited under some goals and the assessment is sensitive to the addition of new indicators, the results must be interpreted with caution. The number of indicators and the availability of data have substantially increased since the Report’s previous edition, thus the results should not be compared with those of previous years.
Current Status (CS) Index
Given a specified SDG target value TV for each indicator I, the values for the current year (Icv) and the year 2015 (I0) can be used to measure the progress made since 2015, in relation to the progress needed to reach the SDG target by 2030. The Current Status Index is constructed as follows.
A metric is developed for each indicator to measure the progress made (represented by the blue bar in figure 1.2), as compared with the entire progress needed from 2015 to 2030.
Denoting indicator values for 2015 and the current year by I0 and Icv and the target value for 2030 by TV, and setting the normalized values of the index 0 and 10 for no progress and full achievement, respectively, the current status index is calculated as:
when a desirable direction (increase or decrease) is clear.
For parity indicators, the value is:
If the region (or subregion) has progressed since 2015, the average overall normalized values under each goal provide an index between 0 and 10. But if the region has regressed, the value is negative and indicates the size of regression.
If the current value for an indicator has already reached or exceeded the target value, the Current Status Index does not need to be calculated and is automatically set to 10.
Anticipated Progress (AP) Index
This index compares predicted (anticipated) progress with targeted progress. By predicting the indicator value for the target year and benchmarking the predicted value against the target value, the index provides a measure of how much progress towards the target will still be required by the end of the target year (2030), assuming the pace of progress is sustained. Denoting the predicted value of indicator I for the target year by It, the anticipated progress index can be computed by replacing Icv with It in formulas in previous section.
The Anticipated Progress Index is only calculated for indicators that are not expected to achieve the target. When the predicted value has already reached or exceeded the target or is expected to reach the target by 2030, the indicator is automatically classified as “will be achieved”.
Based on expected progress, indicators are classified into three predefined achievement levels:
In total, 134 indicators are used to compute the Current Status Index for SDG progress assessment in 2021. Of these, however, 4 indicators did not provide sufficient data for 2030 predictions and were not used for Anticipated Progress Index calculations. When more than one variation for an indicator exists (for example health worker density), all variants are used in calculations. Each variant of an indicator is weighted such that the sum of the weights under each indicator is 1. Finally, a weighted average of the progress indices is computed as a progress index for that indicator.
Disaggregation by sex, location or combination of age and sex was available for 31 indicators. To take disaggregated statistics into account, a vulnerable group for each indicator was identified as the group that had made slower progress than the entire reference population. For instance, if the unemployment rate has decreased by 3 per cent since 2015 among an entire labour force population and this rate is 4 per cent among males and 2.5 per cent among females, then the female group is considered vulnerable. Under each series, the progress is measured as average of progress in vulnerable group and the reference population. By counting for vulnerable groups, progress on each series is adjusted for the progress by the most vulnerable group.
In applying both measures of tracking progress at the indicator level, an acceptance threshold of minimum 2 per cent change was considered for progress/regression. In other words, the change was accepted only if the overall change over the period was more than a 2 per cent increase or decrease (depending on the actual and desired direction of change).
Producing the two measures of progress requires prediction as well as imputation of missing values in the current and previous years. These values were estimated using a weighted regression model that uses time-related weights, assuming the importance attached to the indicator values should be proportional to how recent the data are.
Suppose that n data points are available on indicator I for a given region over a period of T years, and we are interested in estimating the value for the year t.
T=tn – t1 where tn and t1 are the latest and the earliest years, respectively, for which data on indicator I are available. The time-related weights work as multipliers that inflate/deflate the rate of change in each period in proportion to its temporal distance to the target year (t). The time-related weight for the ith data points for a given country/region for estimating indicator values of the year is:
Weights are then incorporated into a regression model used for different indicators. In a few exceptions where the indicator is time-independent, time-related weights were not used (e.g., disaster-related indicators, ODA and other financial aid, etc.).
Setting regional target values
Of 169 SDG targets, only 30 per cent have specific (implicit or explicit) target values. For the rest, this report sets target values using a “champion area” approach. This is based on what has been feasible in the past and optimizes the use of available data. The idea is to identify the top performers in the region and set their average rate of change as the region’s target rate. If we imagine all the top performers for one specific indicator as belonging to one hypothetical area, this can be labelled as the region’s champion area whose rate of change equals the average for the top performers. This can then be considered the target rate for the region. In other words, if the region as a whole can perform as well as its champion area over the 15 years from 2015 to 2030, we should expect to achieve the target value. Subsequently, the universal target value for the region can be derived by applying the rate of change in the champion area to the regional value in the base year. In this report, the regional value is the average value of the indicator over all countries for which data are available. In cases where application of champion area was not possible, the top five performers were identified based on the latest available data the average value for those five countries was used as regional target.
Evidence strength - sufficiency of indicators at goal level
Due to limitations on the availability of indicators, the results aggregated at the goal level are based on a percentage of the total global SDG indicators along with indicators from internationally recognized sources. While the latter are not intended to substitute the former, they shed light on targets where otherwise no analysis would have been possible. Therefore, they are taken into consideration when assessing the completeness of the evidence at the goal level. The strength of the used evidence is thus defined as the following ratio:
Where TGlobal, TUsed and PUsed represent, respectively, the total number of indicators in the global SDG framework, the number of global SDG indicators used in the calculations, and the number of indicators from widely recognized international data sources used.
For ease of analysis, a strength symbol denotes the evidence strength factor according to the table below.
3 Bidarbakht-Nia, Arman. (2020). "Measuring Sustainable Development Goals (SDGs): An Inclusive Approach", Global Policy, Vol 11, Issue 1, Feb 2020, P 56-67. See: https://data.unescap.org/sites/default/files/public/guide/attachments/Measuring_SDGs_ESCAP_SDG_Gateway.pdf.
4 Bidarbakht-Nia, Arman. (2022). ‘SDG Progress Assessment; Comparing Apples with What?’ 1 Jan. 2022: 245 – 250. See: https://data.unescap.org/sites/default/files/public/guide/attachments/SDG_Progress_Assessment_ESCAP_SDG_Gateway.pdf.
5 Bidarbakht-Nia, Arman. (2017). "A weighted extrapolation method for measuring the SDGs progress", ESCAP Working Paper Series. Available from http://www.unescap.org/resources/working-paper-series-sdwp04march-2017-weighted-extrapolation-method-measuring-sdgs.