The Used Car Salesman approach to compliance mapping

 

The Used Car Salesman approach to mapping is the most simplistic, and often acts as a fallback position from the Consulting Army approach. It is based on a “best guess” at what the mapping might be.

When we were at the RSA 2017 conference in San Francisco, we encountered several organizations that provided “mappings” of one Citation to another. When we asked how they achieved the mapping, each said “we hired good people, gave them training, and they just did the work.” When pressed about how they did the work, the answer was always the same “they used their best judgment;” or “Trust me, I’m a pro at this” and nothing more.

The same can be said for the people who do a lot of the work for the government. Take, for example, the very first of the mismatched mappings in the NIST Framework for Improving Critical Infrastructure Cybersecurity. What you’ll get from folks like this is a spreadsheet of crosswalked Citations wherein you’ll see the full Citation from the base source, but only the references from the mapped sources, such as from the NIST document shown here.

NIST ID.AM-1 Physical devices and systems within the organization are inventoried and an inventory of these assets shall be drawn up and maintained.ISO/IEC 27001:2013 A.8.1.1
ISO/IEC 27001:2013 A.8.1.2
NIST SP 800-53 Rev. 4 CM-8

It looks reasonable on the surface. You can probably guess that the three Citations that “match” are about creating a hardware inventory. The problem is, the people who created these “matches” really just read the documents and took a wild guess that they matched.

They didn’t really apply a scientific approach to the mapping, nor did they present any evidence that they did their work. They give us their answer and expect us to trust them based on their degrees, certifications, and who they work for. The problem is, only one of those combinations match.

Once you see the comparison Citation text of all four Citations, you’ll understand why only one of the pairs match. Below is the same table, but this time we’ve added each of the Citations so that you can really compare them.

NIST ID.AM-1 Physical devices and systems within the organization are inventoried and an inventory of these assets shall be drawn up and maintained.ISO/IEC 27001:2013 A.8.1.1 Assets associated with information and information processing facilities shall be identified.
ISO/IEC 27001:2013 A.8.1.2 Assets maintained in the inventory shall be owned.
NIST SP 800-53 Rev. 4 CM-8 The organization verifies that all components within the authorization boundary of the information system are not duplicated in other information system component inventories.

According to the two ISO standards on mapping and harmonization, ISO 704 Terminology work: Principles and methods, and ISO 860 Terminology work: Harmonization of concepts and terms, the only way you can justifiably map one Citation to another is through mapping both the primary verbs and the primary nouns together.

Let’s break each of these four Citations down into their basic noun and verb pairs. We will bracket the nouns with straight brackets “[ ]” and bracket and italicize the verbs with curly brackets “{ }”.

The NIST ID.AM-1 Citation has two mandates. One is to make a comprehensive complete list of physical devices and systems. The other is to record that complete list of devices into a report.

REFERENCETAGGED CITATIONNOUN AND VERB PAIRING
NIST ID.AM-1[Physical devices] and [systems] within the organization are {inventoried} andN: Physical Device or system (e.g., assets held by the organization) V: inventory (e.g., the activity of making a comprehensive complete list of things.)
an [inventory of these assets] shall be {drawn up and maintained}.N: inventory of the assets (e.g., the record produced after tallying) V: drawn up and maintained (e.g., the activity of creating and maintaining something)

Now let’s look at the first of the ISO Citations. The Citation asks us to identify the assets.

REFERENCETAGGED CITATIONNOUN AND VERB PAIRING
A.8.1.1[Assets] associated with information and information processing facilities shall be {identified}.N: Assets V: identified

Do these match? Let’s see. We can reasonably say that a physical device is a type of asset and identifying something is a part of the activity of inventorying. So, yes, they match. Great! They’ve built a bit of trust!

NISTNOUN AND VERB PAIRINGMATCHES?OTHERNOUN AND VERB PAIRING
ID.AM-1N: Physical Device or system (e.g., assets held by the organization) V: inventory (e.g., the activity of making a comprehensive complete list of things.)YesA.8.1.1N: Assets (e.g., assets held by the organization) V: identified (Establish and verifying what something is)

But what about the other ones? Not so much. Once we start looking past the first Citation pairing we see that there is some guessing going on. Let’s look at the second ISO reference as well as the NIST 800-53 reference.

REFERENCETAGGED CITATIONNOUN AND VERB PAIRING
A.8.1.2[Assets] maintained in the [inventory] shall be {owned}.N: Assets (as above), inventory (as record) V: own (taking responsibility for)
CM-8The organization {verifies} that all [components within the authorization boundary of the information system] are [not duplicated] in other information system component [inventories].N: system components (parts of assets), duplicate entries (record entries), inventory (as a record) V: verify (declare as true)

The ISO reference A.8.1.2 does share the noun "assets" with the NIST ID.AM-1 document. However, the verb that it calls for is owning the assets that are in the inventory. This isn’t about identifying assets or about entering their information into a record – it is about the ownership of the assets.

The NIST 800 53 reference (CM-8) shares the noun of asset and the noun of inventory with the NIST ID.AM-1 reference. Verifying the inventory entries might be linkable to the activity of maintaining the record. However, checking for duplicate entries is much more specific than simply maintaining the inventory. It’s a subset at best.

NISTNOUN AND VERB PAIRINGMATCHES?OTHERNOUN AND VERB PAIRING
ID.AM-1N: Physical Device or system (e.g., assets held by the organization) V: inventory (e.g., the activity of making a comprehensive complete list of things.)NoA.8.1.2N: Assets (as above), inventory (as record) V: own (taking responsibility for)
NoCM-8N: system components (parts of assets), duplicate entries (record entries), inventory (as a record) V: verify (declare as true)

Guessing at a Citation to Citation match just doesn’t work. These folks hide their guesswork by not showing you the text of the Citations they are mapping. They don’t show you which nouns and verbs they guessed should match each other. And then, they ask you to “trust their judgment” and “trust their work”.

How much would you trust someone who showed you three examples where only one was a match?