Why ‘GDP calculation’ debates require an inter-sectional process of validation

“Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: ‘There are three kinds of lies: lies, damned lies and statistics..”

–Mark Twain in his own autobiography: The Chapters from the North American Review

Mark Twain’s remark made way back in 1898 is worth echoing in the context of the current dispute on testing the calculation and veracity of India’s official GDP numbers. A recent addition made to the debate includes an official note published by the Economic Advisory Council to the Prime Minister (EAC-PM) which is a ‘point-by-point’ rebuttal to Arvind Subramanian’s working paper claim that India’s GDP growth was ‘over-estimated’ by roughly 2.5% in the post 2011-12 scenario.

In economic analysis and debates, it is important to recognise that any critique of methodological frameworks—in statistical or non-statistical cases—broadly relies on two foundational elements: form and structure. A form-based critique may often counter (or invalidate) the content of say a testable hypothesis or research question (in some cases a problem to deconstruct), while structure-based critiques seem to be mostly concerned with the selection of identified variables (say, either dependents or independents in a model), or on the nature of conclusive evidence drawn from pre-existing theoretical models.

In case of national income accounting—which is more of a statistical process of accounting—the form and the structure of analysis can be designed by a given analyser in the way(s) he wants. In other words, selection of indicators used in studying GDP growth trends for a nation over a period of time can be picked to suit a given hypothesis and then validate it accordingly (for example, if I am convinced that India’s GDP numbers were ‘underestimated’ during a given time period, I can pick only those industrial indicators where growth trends showed higher growth than other indicators over a period of time, and accordingly chose a base year that helps me come to the closest result possible, which is in alignment to the testable hypothesis, i.e., of GDP being ‘underestimated’).

Unless one looks at an underlying growth model and tries to support (or counter) it by further developing an alternative explanation to the research question, any robust critique may hardly take us to any objective certainty on what the actual trends are (so how we design and select indicators remain key to GDP diagnosis).

In recent decades, such a ‘math’ heavy pattern in economic analysis and its discourse has been recognised as the mathiness problem, where, too much of mathematical miscellany (especially in form of statistical jugglery), crowds out actual research motives.

Reading EAC’s response to Arvind Subramanian’s working paper, one gets the impression that the actual purpose (hypothesis) of the authors seems to be aimed more at refuting Subramanian’s research findings entirely—from the very first step—without offering an alternative explanation to the changing composition (and current nature) of India’s growth trajectory.

A critique to Subramanian’s conclusions is surely acceptable from a structural point owing to the ‘extrapolations’ he makes in asserting that GDP numbers were ‘overestimated’ post the change of base year of calculation to 2011-12.

The authors of the EAC note and acknowledge themselves: “GDP calculation is an imperfect art”, and that “essence of the exercise is to pursue perfection in calculations in a manner that best reflects a country’s (economic) situation”. And it is in this latter aspect where the note fails in providing much needed clarity on.

Irrespective of whether GDP numbers remained ‘overestimated’ or ‘underestimated’ (after changes in GDP calculation methodology) can remain a persistent squabble if the underlying growth model remains the same. GDP calculations work primarily on ‘estimates’ made with ‘extrapolations’ from growth-driving sectoral trends based on real-time trends. The real question is whether these ‘extrapolations’ end up reflecting India’s actual economic situation during a period of time.

For this, one can look at doing an inter-sectional analysis or a mapping of GDP figures with other related composite (or structural) indicators reflecting an economy’s performance. National income (or output) is simply represented as ‘Y’, where ‘Y’ is seen as an accumulated function of value added in growth of: ‘C’ or consumption demand (or expenditure); ‘I’, investment demand (or expenditure); ‘G’, government demand (or expenditure), along with ‘NX’, net exports (or trade account position).

To check whether GDP accounting methods depict well an economy’s actual condition, one can simply look at the proportional relationship between growth estimates with these composite aggregates (‘C’, ‘I’, ‘G, and ‘NX’). If the structural composition of growth is changing for any nation (say, from current context of India or as seen in the 1990-2000 period in China), one can understand this by looking at the performance of these compositions (and make changes in the growth model accordingly).

In India’s case from the past three-to-four years (especially since early 2014), well-argued evidence shows how data on these composite aggregates—calculated from real credit growth, export growth performance, vehicle purchases, domestic private investment volumes across sectors etc.—are all seeing a substantive decline in performance. The Union government’s own fiscal expenditure position observed a ‘slippage’ too (when weakening private investment levels were causing public expenditure in certain sectors like in power sector to rise).

Dr. Rathin Roy, an EAC member and one of the authors of the published note, has written and spoken extensively on some of the challenges the Indian economy is facing from a growth perspective (say, based on the disproportionate ends in consumption growth levels while India proceeds towards “middle-income trap” and its fiscal-side witnesses a ‘slippage’).

So, if almost all these indicative aggregates are poorly performing from one quarter to another how is it that the overall GDP numbers as claimed by the government continue projecting sustained growth of above 6.5% for 2014-17? In other words, either ‘extrapolations’ made by official government released numbers have chosen only indicators (from a given sector) where the final growth numbers can reach an above 6.5% mark or there is something structurally wrong in the underlying growth model which make the entire statistical process a mishmash between ‘the GDP growth data’ and the ‘actual conditions of growth’.

Subramanian’s study and methodology can (and must) definitely be questioned and perhaps his actual GDP estimates may be argued to be ‘wrong’ or ‘flawed’ (he himself acknowledges that there are issues within some of the assumptions he makes), but the same can be argued about anyone else crunching these numbers from a similar underlying model without offering a clear explanation on changes seen in the structural composition of India’s growth trajectory (say from 2011-12 onwards).

At the same time, while the debate on GDP calculation methods might continue, a more pressing issue is, what does this do to the actual credibility of Indian government’s own statistical processes or how open are they to independent scrutiny and test? Base years for GDP calculations have been changed previously as well to correct for more accurate depictions of growth and independent analysts (including leading economists) trusted these numbers for most of the time up until now.

The EAC or the Union government may need to acknowledge here that its facing a serious ‘crisis of statistical credibility’, and while Subramanian may be ‘proved wrong’—what scholars require is a ‘trustworthy’ space for knowing whether analysis offered by the government (in any form or structure) can actually be relied upon for further work.

And this means there should be minimal scope for doubt in testing existing methods of growth diagnosis in any independent scrutiny (for example, if I think prices of onions are rising from data available from an inflation measuring index number for a given region, I should be able to go to that particular region and observe whether the macro data-trends stand validated or not from a micro-standpoint).

All policy-discussions and analysis use official GDP growth data provided by CSO that comes with the government’s approval. Conditions of growth in the Indian economy seem to have only worsened in recent years, and amid this, if the ‘basic denominator’ of all policy discussion (GDP) itself paints a disputable picture, no serious scholar or researcher will trust the evidence provided (depending thereafter on privately sourced from companies extracting data for profit).

An intersectional exercise of validation in composite indicators and the underlying growth theory needs an institutional agency (independent of government’s influence) that can help in verification of periodic national income data, and this will go a long way in allowing independent organisations to analyse GDP data as and when released to see if it best reflects the conditions of growth that actually exist. As of now, this isn’t the case.

Views are personal.

Deepanshu Mohan is associate professor of economics, and director, Jindal School of International Affairs,, O.P. Jindal Global University.