Our examiner is asking about the “scope” of our IT audits. What is she referring to, and how do we define a reasonable scope?
Audit results are one of the first things examiners want to see, and the “scope” of the audit is very important to examiners. In fact, the term is used 74 times in the FFIEC Audit Handbook! Scope generally refers to the depth and breadth of the audit, which is in turn determined by the objectives or what the audit is designed to accomplish. The two broad objectives for any audit are control adequacy and control effectiveness*. Control adequacy means that the controls you have in place (policies, procedures and practices) address all reasonably identifiable risks. These audits are sometimes referred to as policy (and sometimes ITGC, or IT general controls) audits. Although the standards used for these audits may differ (more on that later), the scope of these audits should ultimately address the requirements outlined in the 11 IT Examination Handbooks.
Once control adequacy is established, the next thing the examiners want to know is “OK the controls exist, but do the controls work?”, i.e. are they effective? Are they indeed controlling the risks the way they were designed to? Those types of audits are more commonly (and accurately) described as tests or assessments, usually referred to as penetration (PEN) tests, or vulnerability assessments (VA). They may either be internal, external, or (preferably) both.
Sequentially, the audits must be conducted in that order. In other words, you must first establish adequacy before you test for effectiveness. It really doesn’t make sense to test controls that don’t directly address your risks. In fact although an auditor will sometime combine the 2 audits into a single engagement, I encourage folks to separate them so that any deficiencies in control adequacy can be corrected prior to the PEN testing.
One more thing to consider is the standard by which the auditor will conduct their audit, sometime referred to as their “work program”. These are the guidelines the auditor will use to guide the project and conduct the audit. While there are several industry established IT standards out there…COBIT, ITIL, COSO, ISO 27001, SAS 94, NIST, etc., there is no one single accepted standard. The fact is most auditors use a customized hybrid work program, and the vast majority are perfectly acceptable to the examiners. However at some point in your evaluation process with a new auditor you should ask them why they prefer one standard over another. Whatever their preference, make sure that somewhere in their scope-of-work document they make reference to the FFIEC examination guidelines. This assures that they are familiar with the unique regulatory requirements of financial institutions.
Regarding cost, there are often wide disparities between seemingly similar engagements, and it’s easy to see why. In order to make a side-by-side comparison you’ll need to know a few things: Is the audit focused on control adequacy or control effectiveness (or both)? If both, are they willing to break the engagement into 2 parts? What is the audit standard they’ll be using, and why? What methods will they use to test your controls; inquiry or inspection and sampling? Are vulnerability assessments internal or external (or both)? What are the certifications of the auditors and how much experience do they have with financial institutions? Finally, if the examiners have questions or concerns about the auditor’s methodology, or if examiner findings seem to conflict with audit results, will the auditor work with you to respond to the examiner?
In summary, the scope of the audit is defined as either:
- To assess and determine the adequacy (or design and operation) of our controls
- To assess and determine the effectiveness of our controls
- All of the above
So the examiner will want to know the scope, but it’s to your benefit for you to understand it too because examiners will often use the results of your audits to shape and possibly reduce* the scope of their examination!
* Some audits will break the first objective into two sections; design (are they designed properly), and operation (are they in place and operational).
** FFIEC IT Examination Handbook, Audit, Page 8