CIA (e)Bulletin/(e)Bulletin de l'ICA
Past Issues | www.cia-ica.ca  
   
December 2015
 
 

A Few Thoughts on Models

Print Print this Article | Send to Colleague

 

By Pierre Dionne, FCIA

On October 5, the Actuarial Standards Board (ASB) published an Exposure Draft (ED) for Standards of Practice – Use of Models. The proposal would modify part 1000 of the Standards of Practice (SOP) to clarify the duties of actuaries with respect to the use of models, whether internal or external. At the same time, the Practice Council released a Draft Educational Note – Use of Models, to help clarify the new standards for practitioners. Although the deadline for submitting comments to the ASB is now past, the deadline for comments on the draft educational note (EN) is January 8, 2016.

Why a Standard on the Use of Models?

The ED was met with criticism from some areas of practice. It is viewed by some as too onerous to comply with, while others question its utility or relevance. However, the intent of the proposed SOP is not to create additional work for actuaries. Rather, testing models and documenting such tests is considered best practice, and should be done by most actuaries already. I doubt anyone who develops a model on an Excel spreadsheet will use this model without a rigorous check of the formulas for accuracy and reasonableness. The proposed modifications to part 1000 only serve to highlight what actuaries are, or should be, already doing.

The Definition of a Model

The ED defines a model as "a practical representation of relationships among entities or events using statistical, financial, economic, or mathematical concepts. A model uses methods, assumptions, and data that simplify a more complex system." Based on this definition, the EN provides examples of what is or is not a model. Whether a calculation is a model or not hinges, in my view, on the second sentence above, and the fact that a model simplifies a more complex system. To provide further examples, calculating the market value of a bond would not be a model. This follows since the market value of a bond is based on known amounts such as the par value, the maturity date, the coupon rate, and the payment period. However, the standard practice of P&C actuaries to calculate a discount rate based on a bond portfolio by adding assumptions around reinvestment rates and bonds default would be a model as it makes simplifying assumptions around future unknown events.

Proportionality

Both the ED and EN introduce the concept of proportionality, the ED stating that "the strategies to mitigate model risk would depend on the financial significance that an incorrect model run would have on the work." The EN introduces the concept of risk-rating a model. The method presented is but one of the strategies available to an actuary in deciding the degree of review to be done on a model.

It goes without saying that reputational risk to the actuary also features prominently in the degree of review of a model. One would expect that an actuarial evidence actuary would review the model used to calculate the commuted value of a pension for a marriage breakdown three times rather than once before heading to court. Similarly, a Canadian Asset Liability Method (CALM) model would be subject to greater scrutiny given the immediate financial impact of a mistake, as opposed to a Dynamic Capital Adequacy Testing (DCAT) model which deals in identifying future risk.

New Versus Existing Models

Common sense would suggest that a newly developed model would be subject to greater scrutiny than when reusing an existing model. However, this does not mean that an existing model would not be subject to some degree of review. In large firms, models may be accessible by several individuals, and are usually subject to continuous improvements. One should never assume that a model that worked correctly the prior time will work correctly in the current run. Strategies exist to minimize these risks, such as locking all areas of a model except for input areas, and clearly documenting all changes performed to a model. With proper controls in place, the work required to satisfy the ED for existing models should be minimal.

Conclusion

The EN tries to bring a certain level of practicality to the ED. Again, the goal of the ED is not to create work for actuaries, but rather to formalize what should already be happening at firms and companies across the country. It is my opinion that the Modelling Task Force chaired by Bob Howard did a superb job with this first version of the EN. However, I am also convinced that the EN could be improved further. This is where you come in. Please take a few hours to review the EN, and let us know if any areas need more clarification, or whether you disagree (or agree) with any sections of the EN. Please send comments to Bob Howard, Chair, Modelling Task Force at bob@howardfamily.ca or Chris Fievoli, CIA resident actuary at chris.fievoli@cia-ica.ca. We look forward to receiving your comments prior to the deadline of January 8.

Pierre Dionne, FCIA, is Chair of the CIA Practice Council.  

Back to CIA (e)Bulletin/(e)Bulletin de l'ICA

Share Share on Facebook Share on Twitter Share on LinkedIn