By Hassan Rasool<\/p>\n\n\n\n
The amount of data available\u00a0in the world for\u00a0inquiry is increasing exponentially. At the same time, the relevance of evidence-based decision making is also increasing. Many country-level reports use aggregate data for building metrics and indicators to rank countries. Often with the publication of these reports, many media debates occur and we draw different inferences from the same empirical evidence. It is important to understand how to interpret data in such ranking reports and then one builds an argument or advice that is aligned with the philosophy of \u201cevidence based\u201d.<\/p>\n\n\n\n
Given the importance of these indicators in the socio-economic landscape, for better interpretations, the individual researchers, policy makers and opinion leaders need to identify and address several problems with the nature and application of\u00a0these metrics and indicators. The following is an evaluation of different type of indicators. Subsequently, a user guideline is provided for more meaningful analysis and advice.<\/p>\n\n\n\n
The insight evaluation on any of the popular indicators or matrices should follow a sequential list of questions like<\/em>:<\/p>\n\n\n\n There are several unauthentic sources providing fake\u00a0metrics\u00a0about economic and social indicators. Social media helps in fast propagation of such fake\u00a0metrics\u00a0which do not represent the true state of society and economy.<\/p>\n\n\n\n Much cannot be done about s<\/em>ocial media propagation of false metrics, except verifying the source and counter check<\/em>ing. Do not let s<\/em>uch data be used in your analysis.<\/em><\/p><\/blockquote>\n\n\n\n There are several\u00a0metrics\u00a0which are very poor indicators of the target variable(s).\u00a0For\u00a0example, per capita income is a very poor proxy of poverty or absolute amount of debt is often cited as a proxy of poor economic performance to name a few. <\/p>\n\n\n\n Using such indicators in the design of any policy could be catastrophic, always seek better ones or look for supplements to model a more representative matrix.<\/em><\/p><\/blockquote>\n\n\n\n Several, if not all, of the indices being produced by international organizations are miss-understood. This creates a na\u00efve commentary on their findings. For example, Pakistan stands at bottom end of the Global Innovation Index listing. By reading this statement, one would think that Pakistan has very poor scientific output. But actually innovation, as defined in Global Innovation Index (GII) report is \u2018capability to adapt innovative technology\u2019. The construction of GII involves several input and output indicators. Inputs largely relate to human capital, institutions, infrastructure and market and business sophistication. Scientific output relates to indicators like online creativity. The rank of any country in GII then is actually an average of its rank in scientific out puts and governance indicators. Meaning that poor performance on governance indicators will always outnumber unit development in scientific outputs that might be happening in a greater frequency. <\/p>\n\n\n\n In such cases, identify the glitches, normalize the data and start talking about meaningful conversations on the facts that matter.<\/em><\/p><\/blockquote>\n\n\n\n Pakistan ranks 2nd<\/sup> in the world on Out Of School Children (OOSC) numbers and is at 142 out of 144 countries and lowest in South Asia. Interestingly this low rank is because data was based on registered institutions only and also exclude children studying in madrasas. <\/p>\n\n\n\n For evaluating such instances, dig deeper and identify the measurement mechanisms and procedures used to construct such rankings. In first place, look for the definition of the variable under consideration and you will find what is being measured. Now evaluate it on the elixir of prudence.<\/em><\/p><\/blockquote>\n\n\n\n It is very difficult to gauge that what indicator is truly representative. Even GDP and HDI being the most popular ones might fail on a conservative criterion. Sometimes we can reasonably believe that an indicator is functionally representative enough to move forward. Or there may be cases where you cannot ask questions about its legitimacy even, the results need reflection in greater detail, or one needs to identify policy implications. A classic example is of corruption perceptions index (2019) where ranking of Pakistan created a massive media debate. <\/p>\n\n\n\n Here, where perceptions represent reality, any analysis must first dig down to the anatomy of perception, how is it built and progresses over time before posing any analysis on the subject. But, to complete the picture, see the other side as well; i.e., understand what corruption is, how is it manifested in a society and what happens when any state led effort curbs it. Also, what are the initial, latent, and lagged reactions of such interventions.\u00a0Now it is time to look at the results and take position and share your analysis. <\/p>\n\n\n\n Go back to the books, dig deeper in theory, seek empirical evidence, and use common sense to put forth your analysis. <\/em><\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":" Hassan Rasool spells out some common sense guidelines on interpreting data from country level reports. The indicators in these reports are categorized into five types. A brief on how to approach each type for building advice is explained.<\/p>\n","protected":false},"author":2,"featured_media":1199,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1,8,38],"tags":[388,129,386,151,385],"yoast_head":"\n When Indicators Are Fake<\/h2>\n\n\n\n
When Indicators Are Not Really Representative<\/h2>\n\n\n\n
When Indicators Have Design Glitches<\/h2>\n\n\n\n
When indicators Have Measurement Mischiefs<\/h2>\n\n\n\n
When Indicators Are Representative <\/h2>\n\n\n\n