Macro Models for Policy

Background

This post presents an Executive Summary of major findings and recommendations which emerged from the seminar on “Macro Models for Policy and Planning”. PIDE and NIBAF jointly organized this seminar in 2016 at the request of Mr Ishaq Dar and Mr. Ahsan Iqbal. There was wide participation from several ministries and academic institutions.

The goal was to assess the current state of affairs of what models are currently used for policy making. Also to get some ideas of how to move forward. That is, how can we improve models currently in use, and how we can spread the use of empirical and evidence based planning more widely. As a result of the presentations and the discussions, the following points emerged as the action agenda for further progress:

1. Models for output gap are available and in use, at the Finance Ministry, State Bank of Pakistan, and in academia (PIDE-QAU). Each group is working in isolation, and is unaware of the existence of others working on the same problem. Academia is unaware of the needs of the policy-makers, while policy-makers are in general unaware of the need of models. Nor is there much awareness of how current models could be helpful to their policy decisions.
2. Macro models are seldom used and rarely trusted in policy decisions. People prefer to use intuition and experience, rather than formal models and empirical evidence. This is due to two factors:
   1. Flaws and errors in existing models
   2. Misconceptions regarding models and how to use them for policy.

BOTH sides require substantial training; academics need to learn about how to orient theoretical models for practical use. Policy makers need to learn how to utilize formal models and evidence based procedures to help improve decision making. Key obstacles to progress are discussed below followed by recommendations to overcome them.

Major Misconception

We have choice between using models and NOT using models

This idea is wrong. Whenever we make plans for the future, or policy decisions, we MUST use models to forecast and compare likely outcomes. The choice is always between explicit and articulated models versus intuitive and hidden models. Experience and seat-of-the-pant decisions rely on an implicit understanding by decision makers which are not made clear. Such policy and decision-making processes are extremely personalized, erratic, and hit-or-miss. This informal method of policy and decision making, currently in use everywhere in Pakistan, has the following problems:

1. Since the reasons for decision making are not articulated, we can never tell if they were right or wrong.
2. There is no possibility of making progress, or learning from past errors.
3. Decision making depends on judgment calls, which varies from person to person, and is highly erratic.
4. There is no institutional arrangement which puts procedures into place to protect from individual errors and arbitrary decision making. Institutional growth, learning from past experience, does not take place.

Use of articulated reasons for policy making is the essence of transparency. The reasoning behind a decision should be explicitly and clearly articulated whenever it is made. Such reasons include forecasts of expected outcome of policy enactment. Also, what would go wrong if other decisions are taken. This is currently not done since we could easily be held to task for any mistakes if such articulation is provided. All the bureaucratic incentives go in the opposite direction – make decisions without explicitly explaining the reasons for the decisions.

Recommendations

The institutional structure, including bureaucracy and academia, does not encourage or incentivize evidence and model based decision making. Decisions made arbitrarily give freedom to decision makers; do not penalize errors, do not create learning from experience, create dependency on personalities, and do not embed knowledge into institutional structures. Because of this, there is a huge potential for improved policy and decision making in all ministries and departments. But realizing this potential would require coordinated efforts on several fronts, working against inertia, and also changing incentive structures to reward good evidence-based decision making.

PUT IN INSTITUTIONAL STRUCTURE TO REQUIRE ARTICULATION OF REASONS FOR DECISIONS.

Recorded minutes of policy decision making should require clear expression of expected beneficial outcomes – these are always forecasts based on models, either implicit or explicit. Policy makers should be encouraged to express clearly any areas of ignorance. Things they would like to know about how the real world works – in order to make better informed policy decisions. It is this expression of ignorance which will create the demand for research.

ASK POLICY MAKERS TO EXPRESS CLEARLY WHAT INFORMATION THEY NEED IN ORDER TO MAKE BETTER DECISIONS.

For example, when contemplating effects of a export promotion strategy, data on previous parallel episodes and their outcomes, as well as theoretical models which provide guidance on predicted effects would be relevant and important to informed decision making. Policy makers should be encouraged to write down hunches and guesses used to guide their decision-making. These conjectures can be tested later using empirical evidence. For instance, one might guess that depreciation of the Rupee would lead to improvement in the Balance of Payments, increased volume of exports, reductions in imports, but the empirical evidence presented at the conference shows that all of these conjectures are wrong.

ASK ACADEMICIANS TO CLEARLY AND SIMPLY EXPLAIN THE REASONING BEHIND THE POLICY RECOMMENDATIONS MADE USING MODELS.

This should bring out and highlight the assumptions behind the model. The current black box approach is harmful, because effectively, academicians are saying – trust us, we are the experts. However, poor performance of models in crises does not lead to confidence in their performance. Academicians need to open the black box, and explain the reasoning behind their recommendations. Do it in simple terms so field experts can understand (or perhaps reject) that reasoning. Current practice does not create confidence. Modellers say this is the policy recommendation of my model, but all models are wrong, so take it or leave it. They have no skin in the game. They must take responsibility for their recommendations and also explain the logic of the recommendations, instead of putting responsibility on a mechanical model, and thereby avoiding personal responsibility for errors.

AT THE MOMENT, IT SEEMS COUNTERPRODUCTIVE TO USE COMPLEX & SOPHISTICATED MODELS.

This will increase gap and distrust between academia and practitioners. Both sides need to change their practices to bridge the gap and create communication. On the academic side, this involves moving toward simple models based on short causal chains with convincing empirical evidence, rather than fancy black box models.

MODELS HAVE A NEGATIVE ASPECT: THEY INHIBIT OUT OF THE BOX CREATIVE THINKING LYING OUTSIDE THE RANGE OF PAST EXPERIENCES.

Google has created continuous innovations by using the Google 20% policy, which gives employees one day a week to freely innovate on their own, without any guidance. We also have a huge amount of knowledge, experience and talent bottled up inside long time employees, which remains unutilized due to bureaucratic constraints. We should encourage creative expression of new ideas from experiences and new junior staff from time to time. This will create ownership, generate innovative solutions, and allow for learning from experience.

Specific Recommendations

for participating Ministries and Organizations.

MINISTRY OF FINANCE

Budgetary issues are of the greatest importance. Projections of Government Expenditure categorized appropriately, as well as Tax Revenues are of key importance. Foreign exchange needs to be tracked separately, creating a need for models of Exports and Imports. However, this is all basic accounting. For a pro-active approach to management, one needs to creatively assess cost-benefit ratios in different sectors, including sectors not currently in existence. This requires a much higher order of planning. For instance the Korean government created a semi-conductor industry from scratch – there was no way to do accounting for this sector which did not exist.

Similarly, the Ministry of Finance in Japan guided the development of Japan by providing window guidance to the banks, directing the flow of investments to different sectors according to their priority in development. There are many ways to systematize thinking in this way, to get pro-active proposals to direct investment in the future.

MINISTRY OF PLANNING

The largest task is the approval (or dis-approval) of projects requiring financing generated elsewhere. This is a reactive task, responding to demands. A systematic shift to pro-active mode would require some targets for different sectors, and then contacting relevant parties to generate proposals for approval in the target sectors. While this is already being done in an informal way, the relevant cost-benefit analysis will identify the sectors generating the greatest pay-offs. Suitable models and relevant data will aid this planning process to some extent. MoPD&R could then provide the relevant targets to Ministry of Finance, which is in a better position to execute these actions. MORE DETAILED recommendation for MoPD&R are available from MPDR Seminar: Improving Planning and Policy

STATE BANK OF PAKISTAN

Conduct of Monetary Policy requires a fairly good understanding of the mechanisms by which money affects the economy. Unfortunately, many popular theories used for guidance in this respect are empirically invalid. For example,the belief is that lowering exchange rate will worsen balance of payments. But empirical evidence presented at the seminar showed otherwise. Or increasing interest rates will check inflation. But again empirical evidence presented at the seminar strongly suggests otherwise.

General trend in monetary policy is shifting from Keynesian demand management towards Monetary Inflation Targeting; however this shift is not supported by the empirical evidence. We should strive to make evidence based policy grounded in empirical realities; this will require substantial hard work in many dimensions – improvements in models, improvements in data, improvement in understanding of the data and models among the members of the MPC. They require more intensive training in the history and practice of monetary policy, as well as global experiences. The documentation of reasons for Monetary Policy Decisions is an excellent step in the direction of transparency. This is in line with the recommendations being made here.

PAKISTAN BUREAU OF STATISTICS

As different departments shift towards evidence based policy making, they will demand data required for the empirical verification of affects of their policies. The PBS should be able to provide targeted results. For example, evaluations of all large scale government programs and projects should be within the scope of PBS. The Planning Commission is currently concerned with the problem that long-term evaluation of projects is not being carried out after completion. That is because there is no interest on part of executing stakeholders to turn in the PC-V for long term evaluation.

This problem cannot be fixed by punishment – that is, by denying funding for new project to parties which have not turned in PC-V’s since incentives are not aligned. Any party which does its own appraisal, will have incentive to distort the evaluation. Instead, when the project is assigned, a suitable amount of money (like 10%) must be set aside as payment for independent auditors (engineers or academics) for evaluation of the project. Without such evaluations, Planning Commission is operating in the dark. With independent third party evaluations paid for by the Planning Commission itself, useful information is likely to emerge.

MINISTRY OF COMMERCE

Systematic planning requires some amount of forecasting, hand-holding, providing support along different types of dimensions. The idea of free trade is highly misleading. Most industries have developed under protection – the famous infant industry argument. But protection must be strategically withdrawn so the industry grows up, and does not remain an infant.

We can use many other imaginative strategies. For instance, bilateral or trilateral trade agreements, which can save foreign exchange – a crucial objective of trade. A deal with Iran, or with Iran and Afghanistan, in terms of oil versus some commodity of use to Iran, could be mutually beneficial, and seems more politically feasible now than it was before. Pro-active planning requires more than models, but models can provide guidance regarding costs and benefits. They also help evaluate and prioritize among different possible mechanisms and schemes. In these calculations, the academia can provide valuable support to the Ministry.

PIDE-QAU AND HEC

Incentives for academics are not aligned with the need to produce high quality research. The race to publish in impact factor journals is counter-productive in many ways. Research on genuine local Pakistani problems is unlikely to be of interest to high impact factor international journals. Do we want to put our best brains to work on solving European and American problems which will get published in their journals? We should develop our own indigenous impact factor, not based on publication, but based on application. If research is credited with providing policy direction and promoting discussion on relevant and genuine problems of Pakistan, it should receive recognition and reward. This will require out-of-the-box thinking and leadership, since I am not aware of models which could be directly used for this purpose.

Concluding Remarks

There are no built in incentives in the system to change for the better. Furthermore, there is a lot of potential to dramatically improve the system of decision making currently in place. Only one small part involves greater use of systematic modeling procedures. In fact, realizing this potential will require hard work and strategic interventions in many dimensions. Many of these interventions have already been proposed by the Ministry of Planning – in terms of improving management using KPIs and the like. This external structure or body of reforms needs to supplemented and supported by a spirit, based on motivating people to selflessly serve the nation. The creation of this spirit will produce the energy and drive required to overcome the resistance to any reform efforts.

Note: an earlier version of this blog has been posted here: https://worldeconomicsassociation.net/pakistan/2020/04/29/macro-models-for-policy/