Tag: Vendor Management

  • Home
  • Vendor Management
01 Nov 2012
Hot Topics Tom Hinkel 0 comment

FFIEC Updates Technology Service Provider Guidance

Just posted, the new Booklet rescinds and replaces the previous one issued in March 2003, and is the first Booklet replacement since Retail Payment Systems in 2010.  In general this is not so much a complete re-write as a reinforcement of the importance the agency places on strong vendor management, which is a concept that we’ve seen from other recent FFIEC releases and updates.

So with all the similarity between this new publication and the one almost 10 years ago, I think it’s instructive to focus on the differences between the two to see how the FFIEC’s thinking has evolved.  It also allows the institutions affected to know exactly what they need to change or adjust to remain in compliance.

First of all, both Booklets state the following:

“A financial institution’s use of a TSP (technology service provider) to provide needed products and services does not diminish the responsibility of the institution’s board of directors and management to ensure that the activities are conducted in a safe and sound manner and in compliance with applicable laws and regulations…”

and perhaps for extra emphasis the new Booklet adds the following verbiage:

“…just as if the institution were to perform the activities in-house.”

Nothing new here, all institutions are acutely aware that they bear full responsibility for the confidentiality, integrity and availability of their customer’s data regardless of where it may reside.  This re-statement is perhaps insignificant by itself, but interesting when taken in combination with the next sentence:

Old guidance – “Financial institutions should have a comprehensive outsourcing risk management process to govern their TSP relationships.”

New guidance – “Agencies expect financial institutions to have a comprehensive, enterprise risk management process in place that addresses vendor management for their relationships with TSPs.”

What is significant here is the addition of the word “enterprise” to the risk management process, indicating that you must acknowledge that vendors carry multidimensional risks.  These risks include not just operational risk (risk of failure), but strategic risk, regulatory risk and reputation risks as well.

However to me the most significant change in the guidance is in the sentence beginning with “…(the risk management) process should include…”, because this is what the regulators will expect from you.  Compare these:

Old guidance – “Such processes should include risk assessment, selection of service providers, contract review, and monitoring of service providers.”

New guidance – “The risk management process should include risk assessments and robust due diligence for the selection of TSPs, contract development, and ongoing monitoring of all TSPs’ performance.”

It’s clear that regulators will expect much more from your vendor risk management process going forward.  Not simply selecting a service provider, but robust due diligence in the selection process.  Not just contract review, but contract development.  And not just basic monitoring, but ongoing monitoring of all TSP’s performance.

The new guidance goes on to state that federal regulators expect technology service providers to be familiar with, and adhere to, not just this Booklet, but all 11 Booklets in the IT Examination Handbook series.  One more reinforcement that there are not 2 standards of measurement…one for financial institutions and one for vendors…but only one.  And that one same standard will be enforced by the same federal regulators that currently examine you.

The guidance goes on to describe how they will classify service providers (by size and criticality of the services they provide), and how that classification will determine who will examine them, and how often they can expect to be examined.   As far as who can expect to be examined, any service provider that provides any of the following services:

  • Core application processors
  • Electronic funds transfer switches
  • Internet banking providers
  • Item processors
  • Managed security servicers
  • Data storage servicers
  • Business continuity providers

So pretty much anyone that provides an application, system, or process that is vital to the successful continuance of a critical business activity, or anyone that interfaces with a critical business system, can expect to be examined.

Aside from being more comprehensive, the actual examination process hasn’t changed much.  Examiners will still scrutinize the AMDS (Audit, Management, Development and Acquisition, and Support and Delivery) components, and will still assign a 1 through 5 numerical score to each component with 1 representing the highest or best, and 5, the lowest rating or worst.  Examiners will then use the component scores to determine the overall composite rating.  Again, nothing new there.

So in summary, not a drastic change as much as a reiteration with amplification and clarification.  Simply put, more of the same…more regulatory expectations for your vendor management program, which means more scrutiny by the examiners (for you and for your vendors), all of which means more effort on everyone’s part!

08 Aug 2012
Hot Topics Tom Hinkel 0 comment

7 Cloud Vendor Deal Breakers for Financial Institutions

With all the recent focus on vendor management in general, and cloud vendors in particular, there has been a lot of discussion about changing regulatory requirements and best practices.  For the most part, cloud vendors must adhere to the same due diligence, contract, and monitoring guidelines as any other vendor  However there are a few (often overlooked) elements that must be considered prior to engaging any cloud-based vendor.  Elements important enough to be deal breakers if you (and they) can’t answer “yes”.

  1. Do they contractually hold themselves to the same high data privacy, security, confidentiality, integrity and availability standards required of financial institutions?   It used to be understood that anyone offering services to financial institutions had to contractually adhere to GLBA guidelines, but with all the relatively new vendors competing for your business, it can’t be assumed or taken for granted any more.  Make sure the contract stipulates it.
  2. If so, can they document adherence to that standard by producing a third-party report, like the SOC 2?  Even if the contract stipulates adherence, you must determine the adequacy and effectiveness of a servicer’s internal controls by requesting, receiving and reviewing the appropriate third-party report prior to engaging…and then periodically throughout the relationship.
  3. Do you know exactly where your data will be physically stored?  Both the biggest strength and the biggest weakness for cloud vendors is in the redundant and distributed nature of the data.  Having data stored multiple times in multiple locations throughout the country is great for high availability, but makes it almost impossible to ensure compliance with your policies for proper handling and storing of information.  You must know where you data is located at all times, and how it gets there.  And if your data is transmitted or stored outside the U.S., you’ll need to understand the rules and regulations of the hosting country.
  4. Will they retain and destroy information consistent with your internal data retention policies?  Internal retention and destruction policies must be observed regardless of how or where the data is stored.  If the data is stored in multiple locations, are all occurrences destroyed?  There may be additional regulatory and legal exposure if data is either destroyed too early, or retained too long.
  5. What happens to your data once your relationship with the vendor is terminated?  The vendor disengagement process is particularly challenging with cloud vendors because you can’t simply walk away any more than you can just throw out a hard drive.  Is the data irretrievably wiped, or simply deleted?  What about the encryption keys?
  6. Do they have a broad and deep familiarity with the regulatory requirements of the financial industry?  According to the most recent statement from the FFIEC on managing cloud vendors, because of the increased legal and regulatory risks, “managing a cloud computing service provider may require additional controls if the servicer is unfamiliar with the financial industry”.
  7. If so, are they willing and able to make changes to their service offerings necessitated by those requirements? Even if the vendor demonstrates adequate familiarity with the financial industry, are they willing to make the necessary changes in their services if and when regulations change?  Unless financial companies make up the majority of their clientele, they may not be, and “under such circumstances, management may determine that the institution cannot employ the servicer.”
03 Aug 2012
Hot Topics Tom Hinkel 0 comment

Risk Assessing iCloud (and other online backups) – UPDATE 2, DropBox

Update 2 (8/2012) – Cloud-based storage vendor DropBox confirmed recently that a stolen employee password led to the theft of a “project document” that contained user e-mail addresses. Those addresses were then used to SPAM DropBox users.  The password itself was not stolen directly from the DropBox site, but from another site the employee used.  This reinforces the point I made in a previous post about LinkedIn.  If you have a “go-to” password that you use frequently (and most people do) you should assume that it’s out there in the wild, and you should also assume it is now being used in dictionary attacks.  So change your DropBox password, but also change all other occurrences of that password.

But passwords (and password change policies!) aside, serious questions remain about this, and other, on-line storage vendors:

  1. Do they hold themselves to the same high information confidentiality, integrity and availability standards required of financial institutions?
  2. If so, can they document adherence to that standard by producing a third-party report, like the SOC 2?
  3. Will they retain and destroy information consistent with your internal data retention policies?
  4. What happens to your data once your relationship with the vendor is terminated?
  5. Do they have a broad and deep familiarity with the regulatory requirements of the financial industry, and are they willing and able to make changes in their service offerings necessitated by those requirements?

Any vendor that can not address these questions to your satisfaction should not be considered as a service provider for data classified any higher then “low”.

________________________________________________________

Update 1 (3/2012) – A recent article in Data Center Knowledge  estimates that Amazon is using at least 454,400 servers in seven data center hubs around the globe.  This emphasizes my point that large cloud providers with widely distributed data storage make it very difficult for financial institutions to satisfy the requirement to secure data in transit and storage if they don’t know exactly where the data is stored.

________________________________________________________

Apple recently introduced the iCloud service for Apple devices such as the iPhone and iPad.  The free version offers 5GB of storage, and additional storage up to 50GB can be purchased.  The storage can be used for anything from music to documents to email.

Since iPhones and iPads (and other mobile devices) have become ubiquitous among financial institution users, and since it is reasonable to assume that email and other documents stored on these devices (and replicated in iCloud) could contain non-public customer information, the use of this technology must be properly risk managed.  But iCloud is no different than any of the other on-line backup services such as Microsoft SkyDrive, Google Docs, Carbonite, DropBox, Amazon Web Services (AWS) or our own C-Vault…if customer data is transmitted or stored anywhere outside of your protected network, the risk assessment process is always the same.

The FFIEC requires financial institutions to:

  • Establish and ensure compliance with policies for handling and storing information,
  • Ensure safe and secure disposal of sensitive media, and
  • Secure information in transit or transmission to third parties.

These responsibilities don’t go away when all or part of a service is outsourced.  In fact, “…although outsourcing arrangements often provide a cost-effective means to support the institution’s technology needs, the ultimate responsibility and risk rests with the institution.“*  So once you’ve established a strategic basis  for cloud-based data storage, risk assessing outsourced products and services is basically a function of vendor management.  And the vendor management process actually begins well before the vendor actually becomes a vendor, i.e. before the contract is signed.  Again, the FFIEC provides guidance in this area:

Financial institutions should exercise their security responsibilities for outsourced operations through:

  • Appropriate due diligence in service provider research and selection,
  • Contractual assurances regarding security responsibilities, controls, and reporting,
  • Nondisclosure agreements regarding the institution’s systems and data,
  • Independent review of the service provider’s security though appropriate audits and tests, and
  • Coordination of incident response policies and contractual notification requirements.*

So how do you comply (and demonstrate compliance) with this guidance?  For starters, begin your vendor management process early, right after the decision is made to implement cloud-based backup.  Determine your requirements and priorities (usually listed in a formal request for proposal), such as availability, capacity, privacy/security, and price…and perform due diligence on your short list of potential providers to narrow the choice.  Non-disclosure agreements would typically be exchanged at this point (or before).

Challenges & Solutions

This is where the challenges begin when considering large cloud-based providers.  They aren’t likely to respond to a request for proposal (RFP), nor are they going to provide a non-disclosure agreement (NDA) beyond their standard posted privacy policy. This does not, however, relieve you from your responsibility to satisfy yourself any way you can that the vendor will still meet all of your requirements.  One more challenge (and this is a big one)…since large providers may store data simultaneously in multiple locations, you don’t really know where your data is physically located.  How do you satisfy the requirement to secure data in transit and storage if you don’t know where it’s going or how it gets there?  Also, what happens if you decide to terminate the service?  How will you validate that your data is completely removed?  And what happens if the vendor sells themselves to someone else.  Chances are your data was considered an asset for the purposes of valuing the transaction, and now that asset (your data) is in the hands of someone else, someone that may have a different privacy policy or may even be located in a different country.

The only possible answer to these challenges is bullet #4 above…you request, receive and review the providers financials and other third-party reviews (SOC, SAS 70, etc).  Here again, large providers may not be willing to share information beyond what is already public.  So the answer actually presents an additional challenge.

Practically speaking, perhaps the best way to approach this is to have a policy that classifies and restricts data stored in the cloud.  Providers that can meet your privacy, security, confidentiality, availability and data integrity requirements would be approved for all data types, providers that could NOT satisfactorily meet your requirements would be restricted to storing only non-critical, non-sensitive information.  Of course enforcing that policy is the final challenge…and the topic of a future post!  In the meantime, if your institution is using cloud-based data storage, how are you addressing these challenges?

* Information Security Booklet – July 2006, Service Provider Oversight

10 Jul 2012
Hot Topics Tom Hinkel 0 comment

FFIEC issues Cloud Computing Guidance

Actually the document is classified as “for informational purposes only”, which is to say that it is not a change or update to any specific Handbook and presumably does not carry the weight of regulatory guidance.  However, it is worth a read by all financial institutions outsourcing services because it provides reinforcement for, and references to, all applicable guidance and best practices surrounding cloud computing.

It is a fairly short document (4 pages) and again does not represent a new approach, but rather reinforces the fact that managing cloud providers is really just a best practices exercise in vendor management.  It makes repeated reference to the existing guidance found in the Information Security and Outsourcing Technology Services Handbooks.  It also introduces a completely new section of the InfoBase called Reference Materials.

The very first statement in the document pretty well sums it up:

“The (FFIEC) Agencies consider cloud computing to be another form of outsourcing with the same basic risk characteristics and risk management requirements as traditional forms of outsourcing.”

It then proceeds to describe basic vendor management best practices such as information security and business continuity, but one big take-away for me was the reference to data classification.  This is not the first time we’ve seen this term, I wrote about examiners asking for it here, and the Information Security Handbook says that:

“Institutions may establish an information data classification program to identify and rank data, systems, and applications in order of importance.”

But when all your sensitive data is stored, transmitted, and processed in a controlled environment  (i.e. between you and your core provider) a simple schematic will usually suffice to document data flow.  No need to classify and segregate data, all data is treated equally regardless of sensitivity.  However once that data enters the cloud you lose that control.  What path did the data take to get to the cloud provider?  Where exactly is the data stored?  Who else has access to the data?  And what about traditional issues such as recoverability and data retention and destruction?

Another important point made in the document, and one that doesn’t appear in any other guidance,  is that because of the unique legal and regulatory challenges faced by financial institutions, the cloud vendor should be familiar with the financial industry.  They even suggest that if the vendor is not keeping up with regulatory changes (either because the are unwilling or unable) you may determine on that basis that you cannot employ that vendor.

The document concludes by stating that:

“The fundamentals of risk and risk management defined in the IT Handbook apply to cloud computing as they do to other forms of outsourcing. Cloud computing may require more robust controls due to the nature of the service.”

And…

“Vendor management, information security, audits, legal and regulatory compliance, and business continuity planning are key elements of sound risk management and risk mitigation controls for cloud computing.”

…as they are for all outsourced relationships!

04 Jun 2012
Hot Topics Tom Hinkel 0 comment

5 Keys to Understanding a SOC 2 Report

Although I have written about these relatively new reports frequently, and for some time now, it still remains a topic of great interest to financial institutions.  Fully 20% of all searches on this site over the past 6 months include the terms “SOC” or “SOC 2”, or “SAS 70”.  Some of this increased interest comes from new FFIEC guidance on how financial institutions should manage their service provider relationships, and some of it comes from financial institutions that are just now seeing these new reports from their vendors for the first time.  And because the SOC 2 is designed to focus on organizations that collect, process, transmit, store, organize, maintain or dispose of information on behalf of others, you are likely to see many more of them going forward.

Having just completed our own SOC 2 (transitioning from the SAS 70 in the previous period), I can say unequivocally that  not only is it much more detailed, but that it has the potential to directly addresses the risks and controls that should concern you as the recipient of IT related services.  But not all SOC 2 reports are alike, and you must review the report that your vendor gives you to determine its relevance to you.  Here are the 5 things you must look for in every report:

  1. Products and Services – Does the report address the products and services you’ve contracted for?

  2. Criteria – Which of the 5 Trust Services Criteria (privacy, security, confidentiality, availability and data integrity) are included in the report?

  3. Sub-service Providers – Does the report cover the subcontractors (sub-service providers) of the vendor?

  4. Type I or Type II – Does the report address the effectiveness of the controls (Type II), or only the suitability of controls (Type I)?

  5. Exceptions – Is the report “clean”?  Does it contain any material exceptions?

Before we get into the details of each item, it is important to understand how a SOC 2 report is structured.  There are 3 distinct sections to a SOC 2 (and they generally appear in this order);

  1. The Service Auditors Report,
  2. The Managements Assertion, and
  3. The Description of Systems.

So simply put, what happens in a SOC 2 report is that your service providers’ management prepares a detailed description of the systems and processes they use to deliver their products and services to you, and the controls they have in place to manage the risks.  They then make an assertion that the description is accurate and complete.  Finally, the auditor renders an opinion on whether or not the description is “fair” as to control suitability (Type I) and effectiveness (Type II).

Products and Services

The first thing to look for in a SOC 2 report is generally found in the Management’s Assertion section.   It will state something to the effect that “…the system description is intended to provide users with information about the X, Y and Z services…”  You should be able to identify all of your products and services among the “X”, “Y”, and “Z”.  If you have a product or service with the vendor that is not specifically mentioned, you’ll need to satisfy yourself that the systems and processes in place for your products are the same as they are for the products covered in the report.  (You should also encourage the vendor to include your products in their next report.)

Criteria

The next thing to look for is found in the Service Auditor’s Report section.  Look for the term “Trust Services Principles and Criteria”, and make a note of which of the 5 criteria are listed.  The 5 possible covered criteria are:  Privacy, Security, Confidentiality, Integrity and Availability.  Service provider management is allowed to select which criteria they want included in the report, and once again you should make sure your specific concerns are addressed.

Sub-service Providers

The next item is also found in the Service Auditor’s Report section, and usually in the first paragraph or two.  Look for either “…our examination included controls of the sub-service providers”, or “…our examination did not extend to controls of sub-service providers”.  The report may also use the terms “inclusive” to indicate that they DID look at sub-service providers, or “carve-out” to indicate that the auditor DID NOT look at the controls of any sub-service providers.  These are the service providers to your service provider, and if they store or process your (or your customers) data you’ll need assurance that they are being held to the same standards as your first-level service provider.  This assurance, if required and  not provided in the SOC 2, may be found in a review of the sub-service provider’s third-party reviews.

Type I or Type II

As with the older SAS 70, the new SOC 1 and SOC 2 reports come in two versions; a Type I, which reports on the adequacy of controls as of a single point in time, and a Type II, which reports on both control adequacy and effectiveness by evaluating the controls over a period of time, typically 6 months.  Clearly the Type II report is preferred, but because the SOC 2 audit guides were just released last year, most service providers may choose to initially release a Type I.  If your concerns about the service provider include whether or not their risk management controls were both adequate AND effective (and in most cases they should), make sure they immediately follow up the Type I with a Type II.

Exceptions

Finally, scan the Service Auditor’s Report section for verbiage such as “except for the matter described in the preceding paragraph…”, or “the controls were not suitably designed…” or “…disclaim an opinion…”, or terms such as “omission” or “misrepresentation” or “inadequate”.  These are an indication that the report could contain important material exceptions that would be cause for concern.

One more thing…pay particular attention to a sub-section (usually found in Description of Systems section) called “Complementary End-User (or User-Entity) Controls”.  This is not new to the SOC reports, the SAS 70 had a similar section, but it is one of the most important parts of the entire report, and one that is often ignored.  This is a list of what the vendor expects from you.  Things without which some or all of the criteria would not be met.  This is the vendor saying “we’ll do our part to keep your data private, secure, available, etc.,  but we expect you to do a few things too”.  It’s important that you understand these items, because the entire auditor’s opinion depends on you doing your part, and failure to do so could invalidate some or all of the trust criteria.  By the way, you should be able to find a corresponding list of these end-user controls repeated in your contracts.

The lesson here is that vendor third-party reviews like the SOC 2 are no longer a “check the box and be done” type of exercise.  As part of your vendor management process, you must actually review the reports, understand them (don’t hesitate to enlist the help of your own auditor if necessary), and document that they adequately address your concerns.

25 Apr 2012
Hot Topics Tom Hinkel 0 comment

FDIC Supervisory Letter Issued on Critical Service Provider

(NOTE:  Although the vendor in question has been publicized by the NCUA, I will not name it here because it is not relevant.  If you currently contract with the vendor you know who it is, and you need to know how to respond to the letter.  If you don’t, you’ll need to know how to respond in case it happens to a critical vendor of yours at some point.)

What if you received this letter from the FDIC on one of your most critical service providers (summarized and redacted)?

“Dear Board of Directors,

Enclosed is a copy of the Information Technology (IT) Supervisory Letter based on the interim review of (your vendor).  We are sending you this Supervisory Letter for your evaluation and consideration in managing your vendor relationship…I encourage you to review the Supervisory Letter as it discusses some regulatory concerns that require corrective action by (your vendors’) management and Board of Directors.

Sincerely,

FDIC Regional Director”

The letter states in part:

“(Vendors’) Executive Management supervision and control over the Risk Management (RM) and Information Security (IS) functions are unsatisfactory. Additionally, the Board of Directors (BOD) does not provide sufficient direction and oversight for management responsibilities, as well as for independent review in these areas by Internal Audit (IA). The breadth and severity of weaknesses noted at this IR stem from management’s failure to adequately address previously identified systemic issues and to take proactive measures to mitigate the identified systemic risks. These weaknesses had exposed serviced financial institutions to increased risk, and have raised concerns regarding management’s ability to establish and enforce effective information security measures commensurate with the needs of (vendor).”

So the FDIC conducted an IT Examination on the service provider.  Nothing new there…IT service providers are subject to the same regulatory oversight as financial institutions, and even have their own Examination Handbook*.   However, in this case the exam uncovered significant material weaknesses in their audit, management and IT controls.  Weaknesses so severe that the FDIC felt it necessary to proactively notify all institutions under their regulatory responsibility that utilize the provider.

Since the FDIC stated that they are sending the letter for “your evaluation and consideration“, they clearly expect you to take specific action on this matter.  Don’t be surprised to see them asking for your formal response during your next visit from them.  So here is what you’ll need to do:

  • The first thing you’ll want to do is call a meeting with the group you use to manage your vendor relationships.  If you haven’t assigned vendor management responsibility to a management committee (as opposed to an individual), do so.  IT Steering or Audit is a logical choice.  Formally document in the committee that “the examiner’s letter represents certain concerns that will cause us to reevaluate the vendor, reassess the residual risk, and consider implementing additional compensating controls”.
  • Request, review and evaluate the vendor’s response to the examiners letter.  Determine whether the response is sufficient to address your concerns.  If not, consider implementing the following additional compensating controls:
  1. Accelerate the normal annual due diligence process by requesting more frequent financial statements (quarterly instead of annual).
  2. Request that vendor provide additional 3rd party security reviews other than SSAE 16 if possible (i.e. SOC 2, PEN tests, etc.).  The SOC 2 is a good choice, as it directly addresses controls related to privacy, security, confidentiality, integrity and availability…all the things that are important to you.
  3. Have legal review existing vendor contracts for possible breach of contract claims.
  4. Consider adding a “right to audit” clause in future contracts.
  5. Become active (or more active) in vendor user groups.  The intent is to stay close to the situation, and possibly influence them to release additional 3rd party reviews (such as SOC 2).

It is important to take action even if you are in a long term contract with the vendor, or if the vendor would be difficult to replace.  And you can’t take the position that since you can’t control what the vendor does, you’ll simply have to go along…that it’s not your problem to solve.  Guidance makes it clear that “institutions should ensure the service provider’s physical and data security standards meet or exceed standards required by the institution.”  So for all intents and purposes, the vendor’s deficiencies are your problem.

*According to the FFIEC:

The federal financial regulators have the statutory authority to supervise all of the activities and records of the financial institution whether performed or maintained by the institution or by a third party on or off of the premises of the financial institution.

The decision to examine a service provider is at least partially based on the number of Bank Service Company Act (BSCA) filings the regulators receive on the provider.  I explain this here, and make the point that because the definition of a “Service Company” has expanded, more service providers can expect more examinations in the future.

© 2024 Safe Systems, Inc. All rights reserved. Compliance Guru is a registered trademark of Safe Systems. Privacy Policy