Developing a Meaningful Scoring System for Risk Assessments

One of the most daunting challenges in building a Risk Matrix may be determining the scoring system to use for your assessment. Then again, getting consensus between all your stakeholders of the probability of an event occurring or agreeing on the impact if it does occur, can even be tougher.

Determining the exact probability and measuring the financial impact of a risk event is difficult, especially if that event has never occurred.  It could be said that you are attempting to measure the un-measurable

Now consider the scoring system in your matrix should attempt to remove subjectivity even though attempting to measure a future event will always have an element of subjectivity in it. Having scoring of High, Low or Medium is most likely not going to meet the auditors needs in the future (even if they have approved such a rating in the past!). If you elect to (continue to) use a scoring system of H-M-L, then you should define a range of what each of the H-M-L scores mean. The important thing is to build logic into your scoring system that will allow you can obtain easier buy-in between your board and auditors. Removing as much subjectivity as you can from your scoring system will also contribute to easier buy-in from your stakeholders (including the auditor).

You may also want to consider using a numbering system based on 1-9 for a more precise scoring mechanism. It will also give you more options for your logic and usually facilitates removing some subjectivity. Which in turn should allow you to gain faster consensus among your stakeholders.

Determining the probability of an event occurring will be difficult if put in terms of percentages. How are you going to determine and agree that the probability of an event occurring is 8% or 9%? Therefore, consider using a probability scoring system based upon likelihood that the risk will occur in a timeframe (will occur in a year, will occur in a 5 year horizon, etc). Again, easier to gain consensus.

Determining the impact of a risk event on an organization can also often very difficult, even if the event has already occurred.  There are so many factors to consider, there is operational risk, reputational risk, human factors and numerous other risk factors. The timeframe of the impact can also be long lasting making the calculation of the impact even more arduous.  So instead of calculating a monetary value for your impact scores, consider scores based upon relative importance to a risk category to the organization. Although it may be difficult to gain consensus of a monetary amount, you may want to include in the score the action if that risk occurred. Would you handle it internally, would you need to report the event to the board? Would you need to report the event to an external body (like on your 10k)? Although it may be difficult to put a dollar value on the impact, you will most likely be able to gain consensus of how the event would be handled, internally versus reported to the board or beyond.

The risk probability and impact ratings for an event help show that all risks are not created equal, some are more important than others. The same logic applies to controls; some controls are more effective than others. A directive control (policy) is not as effective as a detective or preventative control. Automated controls are more effective than manual controls. All these factors need to be applied to your Control scoring system. Lastly, you need to determine if your control is considered best in class or potentially a meager attempt at mitigating the risk.

Now once you have determined a meaningful scoring system and rated all your risks, how do you determine where to spend your limited resources? Do you address risks with a Risk score higher than a particular number (40 or 50 or 60)? How do you determine that those high Risks are adequately being mitigated? There are only a very few risks where you can totally remove the probability or impact of a risk event.  Most likely, you are already addressing your highest risks to some degree. Besides, the risks that are most likely to occur are the ones with the highest residual risk score (after your controls are applied). This is typically where your controls in place are less than best-in-class. Therefore it is important that your scoring system should not only include the probability of a risk occurring, the impact, but also include a scoring mechanism that includes the controls you have in place and the response you have in place if it does!

In conclusion, it is certainly easier to develop a simple matrix and use scores based on High, Medium and Low and try to convince yourself and the auditor that you are doing the best you can to mitigate risks. However, what you really need to do is develop a system that will allow you to build a better organization, not just something that you can use to check off another box that the task is complete.

Our Team
Our team is comprised of experienced executives, managers and consultants who will assist your banking organization in the development, implementation and execution of comprehensive risk management and compliance strategies.  From the initial passage of  Sarbanes-Oxley in 2002, Visage has provided solutions to a client base ranging from private, entrepreneurial companies to large multinationals.