| Contents | Previous | 6.1 Performance Assessment | Next |
The effectiveness of the training will be assessed at two different levels: the individual learners and the organization.
Individual employees and volunteers who take part in this training effort will not be evaluated on a comparative scale. The objective of this training is not to rank employees based on their understanding of the threats and the defensive procedures.
The two main goals are:
helping learners to gain a growing appreciation for the threats, leading them to favor caution in their digital interactions and consistently avoid behaviors that put the organization at risk, and
gaining mastery, through regular practice, of the skills required to fight off phishing scams and other digital threats.
A change in attitude is notoriously hard to evaluate, since answers provided to a survey only partially predict the preferred behaviors when learners are faced to the actual situation. The most accurate assessment would thus involve observations of learners over extended periods of time. This would prove both costly and intrusive in this case, since phishing emails come at unpredictable times and are mixed with private communication.
We would thus recommend to evaluate this change of attitude informally, through interviews with learning champions at the level of each department, rather than trying to quantify it for each individual learner. We will describe the learning champions and their recruitment in more details in the Learning Journey.
Individual mastery of anti-phishing skills will be evaluated by sending made-up scam emails at random intervals during the year. These emails will be sorted in 3 levels of difficulty:
All learners will start with a majority of emails from level 1. Learners who fall into the trap set by an email will be redirected to remedial information, earning useful information instead of putting the organization at risk. These errors will be treated as an integral part of the learning process, and no-one shall be held accountable for failing these tests. The remedial learning material may be read immediately, or downloaded and printed for future reference.
After successfully avoiding traps of a certain level about 5 to 10 times, learners will get upgraded to the next level of challenge, with a majority of emails at that new level of difficulty.
We propose to set the following goals after one year:
While individual learners will be tested through carefully doctored emails, crafted using predictable techniques, the organization as a whole does not have that luxury. In order to protect itself against upcoming threats of growing complexity, it must be evaluated against yet unexpected threats.
This will be done by hiring external consultants, called penetration testers, or pen testers in short, to try its defenses and report any vulnerabilities they identify, without taking advantage of it. We recommend to work with certified penetration testers part of a well-established cybersecurity agency, to be selected at a later stage of the project.
This evaluation shall be done at least once per year, with further recommendations being implemented and deployed progressively, based on the severity of the threat, over the next 6 months.
In addition, an internal committee shall evaluate the actions led during the year to raise awareness of issues related to digital threats. A one page summary of this evaluation shall be included in the annual report of the organization published online for donors and the general public.
| Contents | Previous | 6.1 Performance Assessment | Next |