Let The Robot Do It: Automated Model Risk Documentation Saves Time And Money

Subscribe to Our Blog

Paperwork is a nettlesome fact of life in financial services, especially when it comes to the documentation behind risk models. The Federal Reserve’s 2011 guidelines require a lengthy model risk management (MRM) report that explains in great detail how a model was conceived and built so that anyone at the lender can read it and replicate the model.

Conscientious bankers strive to meet the spirit of the regulations, too. These MRM reports are evidence to the regulators that you’ve built a governance process to ensure that your models are well understood and that you can demonstrate that they’re being used properly.

For most banks, the work of documenting a model stands in stark contrast to the work of building a model. Building a model is both art and science and involves an intuitive understanding of a bank’s business needs and customer base. Documenting a model is, in contrast, is just plain tedious. It’s not unusual for an MRM report to run 150 pages long and take three to six months to write. The process is so painfully slow that we’ve even heard of situations where the documentation took so long that the model became outdated before it could be launched. The underwriters had to start from scratch.

ZAML helps you turn the back-half of the modeling process from a time-consuming chore into a simple push of the print button.

A better idea: Automate the documentation instead. When we designed our Zest Automated Machine Learning (ZAML) software tools, we made sure that auto-documentation (AutoDoc) was woven into the fabric of building credit risk models. By automatically and consistently documenting a model during its construction, ZAML helps you turn the back half of the modeling process from a time-consuming chore into a push of the print button. The MRM document you get is mapped to each of the requirements laid out in the Fed guidelines that everyone follows.

Here are a few reasons why AutoDoc makes so much sense:

Stronger, faster, smarter

A credit risk model is only as good as its data, and with swiftly changing market conditions and customer behavior, that data can get old faster than you might think. AutoDoc means your company can adjust models to reflect evolving conditions more quickly than competitors who are stuck doing old-fashioned documentation.

AutoDoc speeds up the process by producing the required narrative about the model’s inner workings and mapping those points to the specific regulations governing each item. ZAML can even automatically provide the code along with the narrative. This automated process facilitates faster review and tweaking of completed portions of the model because when anything is changed, the documentation updates too, reducing documentation, validation and review time from three months (in a well-designed traditional model) to two weeks with ZAML.

Built for scrutiny

There is no bright line from the Federal Reserve about what constitutes appropriate MRM documentation. Regulators expect more from a bigger bank than from a smaller bank but there’s a huge gray area in between where it’s hard to say exactly how much is enough. AutoDoc allows banks of any size to avoid falling afoul of regulators by providing the most comprehensive level of documentation for every model built on ZAML, regardless of the size of the portfolio it’s handling. Any sized institution can do what the largest banks can do while avoiding getting into debates with regulators about how specific documentation should be.

Allows employees to do what they love

Right now, most everyone except the biggest banks with armies of techies conducts model documentation and validation by hand. Quite honestly, the quants who are programming the model hate having to document every calculation they make each day during the model-building process. It’s these kinds of mundane tasks that keep expert model builders and validators from doing what they love. The manual approach also opens the whole process up to human error, such as transposing numbers into the log book or skipping a step in a calculation. While ML may complicate things by adding more variables and new math, the documentation process in an ML modeling environment doesn’t have to be more time-consuming. In fact, it can be less of a chore.

Automating documentation makes deploying a more powerful ML model more efficient and shifts the burden to the machine, freeing people to be more creative and productive.

And isn’t that what we all want out of life?

Photo by Franck V. on Unsplash