I’ve been reading an excellent paper on introducing, implementing, and measuring the effectiveness of transparency metrics for software systems. And while reflecting on it, I realized that the principles and methodology discussed are applicable for the pharmaceutical compliance space too. While the paper concedes the obscure nature of transparency, it establishes that metrics for transparency can work as a strong ‘accountability mechanism.’ The paper’s author proposes 8 metrics that measure a system’s transparency.

 1.       Accuracy: This measurement makes a comparison between the information provided and the ‘real process of the system.’ If there’s a match, there is a high degree of accuracy. In the context of pharmaceutical analytics, compliance monitoring mechanisms achieve this for us with the help of triggers and KCIs.

2.       Currentness: Measuring gaps between the frequency of data refresh/data upgrade will be a first-level indicator. When compared with other variables, such as the volume of data, the number of people feeding data into the system,

3.       Conciseness: The Baymard Institute recommends 50 to 60 characters per line for optimal readability, which is to imply that brevity is better. The same rule applies when publishing transparency data.

4.       Detailing: This metric is about context. It provides users with information about policies on data (the ‘why’ and ‘how’ of data collection), processing, and data usage. Naturally, this will be qualitative, but with enough of a data repository, the data can be structured and categorized.  

5.       Readability: This isn’t quite the same as ‘portability’ or ease of access. This has to do with system compatibility. If your data security measures leave the end-user device unable to unencrypt data; or if there are issues like language barriers, no data dictionaries, and so forth, you are creating a barrier. Thus the appropriate transparency metric is to measure how easily users can read the data you need to share.

6.       Availability: This is described as the ‘number of interactions needed to access the system.’

7.       Portability: We need to treat data portability in the context of accessibility or how easily that data can be retrieved by users. This isn’t just about the ‘hardware’ side of compliance data. It’s also about the format in which it is created, stored, and moved. That takes in issues of structure (the more structured, the better), the openness of the format, and data presentation. Restrictions on data accessibility need to be clearly defined.

8.       Effectiveness: Is the system achieving its desired goal? There’s an oft-confused conflict between efficiency (minimizing resources used in achieving the system’s goal) and effectiveness (checking if the goal has actually been achieved).

 Individually, each metric tells a story. But what is the collective message they’re giving out about the pharmaceutical manufacturer’s compliance program? Also, how do we translate this into an existing analytics program?