Beacon Economics Rips Sacramento State Study on Regulatory Costs

Recently, Dean Sanjay Varshney of Sacramento State has made a big splash with a new bi-annual economic forecast for Sacramento and some high profile consulting reports on economic policies. I like competition and have refrained from commenting on his new Sac State forecast or any of his consulting reports... until now.

On Monday, the Sacramento Bee ran an op-ed by Chris Thornburg and Jon Haveman from Beacon Economics that ripped Varshney's new report on the cost of regulations on small business in California. Varshney claims regulations cost the California economy $500 billion per year. Beacon says,

Disturbingly, this result is one of the worse examples of schlock science we've ever seen. And chillingly, it is being held up by a number of state leaders... This is truly bad analysis, and it matters because it once again points to the urgent need to have policy research vetted through third parties for methodological soundness and intellectual honesty... The state must make it a priority to base policy decisions on rigorous research rather than pseudoanalysis that twists or flat-out ignores the truth.

Ouch. That's about as harsh as it gets without resorting to personal attacks. However, I actually sent Thornburg and Haveman an email with the title, "You guys are too kind." I wasn't commenting on their manners, but the fact that their op-ed only explained half of the problem with the Sac State study.

The Sac State study has 2 parts: 1. a regression estimating Gross State Product as a function of indicators from Forbes magazine and 2. enters the loss in GSP estimated from this regression into an input-output model (IMPLAN) to get multiplier effects. The Beacon op-ed accurately explains the problem with the first part, but not the second (probably due to word limits or an attempt to keep it simple). The error in the 2nd part may be even worse. This is one of the worst abuses of IMPLAN I have ever seen.

IMPLAN is a very useful input-output economic model. I use it all the time. However, it is also the most misused model in economic research. Because it is relatively cheap and easy to use (as far as economic modeling software goes), there are a lot of people out there doing bad consulting with it. IMPLAN studies often double count impacts and fail to make proper adjustments when the modeling or data assumptions don't hold. Many credible academics won't even use input-output models at all because of these notorious pitfalls. In this case, it is simply inappropriate to use IMPLAN at all since the first step of Varshney and Associates regression is econometrically (I use the term loosely here) estimating a macro output variable (GSP) directly. Even if you trust the first part of their study (and you shouldn't), the 2nd stage is simply double and triple counting the effects of the first stage.

I think if we had a Middle Ages style plague that killed 20% of the state's population, the impact would probably be less than Varshney's $500 billion estimate of regulatory effects. Yes, regulations are costly, but exagerrated studies like these are very destructive to real policy debates. The KQED blog gives a little of the back story on the study here.

This is the 2nd time in a few months that Varshney and Associates have issued a report with exagerrated costs of regulations. A few months ago, they released this report commisioned by a small business group stating that AB 32 (greenhouse gas reduction policy) will cost the California economy over 1.1 million jobs (more than have been lost in the current recession, except the AB 32 effect is permanent). I agree with them that AB 32 regulation will have costs on the CA economy, but Varshney's job loss and cost numbers are unbelievably large and come from poor research methods (I'll spare you the details here).

I have nothing personal against Dean Varshney, but I hope his public policy consulting business fails. Like a good businessman, he knows what his clients want to hear and delivers, but these reports are very misleading. I recommend his clients give me a call. I am confident our Center can produce a better study for a lower price. You may not get the number you want, but it will stand up much better to review.

Postscript:
In an earlier post, I scored Beacon Economics the winner in a debate with the Milkin Institute over manufacturing jobs.

Update (Oct. 15):
In a complete coincidence, 12 hours after I posted this, Dean Varshney walked into my office at Pacific and introduced himself. He was on campus for another meeting, and just wanted to take a moment to meet me and suggest that we might collaborate on a project in the future. He was very nice and obviously had no idea that I just posted this criticism. I was surprised and felt a little guilty inside, so I briefly said nice to meet you too and exchanged small talk. In hindsight, I probably should have politely told him that I think he is working on interesting and important topics, but his research has serious problems.

I like him, and I have heard from others that he is a good business school Dean. Like many successful executives, he appears to be very good at building relationships and creating opportunities. That said, his economic research products need a lot of help. On the other hand, I'm pretty good at the research, but need to work on public relations. Maybe we would make good business partners :)