The setup for your compiled indicator has three parts; you’ll work from the big picture, then zoom into the smaller details. 

1. Name the multidimensional situation: First you’ll document the situation that your organization works in. We’ll call this the “big picture” and discuss it in the next section.
2. Establish indicators: Next, you’ll break the big picture into different dimensions that indicate how the problem arises.
3. Designate individual datasets for each indicator: Finally, for each indicator, you’ll identify a collection of datasets. These datasets should represent a diverse mix of quantitative, qualitative, and/or historical data whenever possible.

Using your compiled indicator 

Once you’ve assembled your compiled indicator, you’ll begin using it by inverting the set-up process. You’ll start with filling in the individual datasets, then seeing how they affect the indicators, and finally assembling the numerical compiled indicator itself. 

Here’s an overview of the process:

1. Gather the designated datasets: Inventory the datasets you have on-hand and identify any gaps in what you need to build a complete inventory of relevant datasets.
   
   There may be gaps where the data exists, but you don’t have access to it (yet). This could take some digging and asking around. The datasets you need could be in the data repositories of your or other organizations.
   
   Sometimes, you’ll find that the gap is there because the data you need hasn’t been gathered yet. In this situation, decide whether you can or should substitute in a thematically similar dataset to which you have access. Beware of the pitfalls of confirmation and searchability biases as you make this decision, and balance those against the time and resources you have for discovery work. 

If you don’t substitute in a similar dataset, go back to the Discovery phase to research and produce the data you need. Remember: Discovery doesn’t have to set back your timeline very much at all, as long as you maintain a tight scope on it. Good, precise data is worth the few weeks it might take to gather it.

If you don’t have time for Discovery, move forward with the data you do have access to, but document your choices so that you create both a path for future leads to go back and collect the data you would have preferred to have and to ensure that you’re building a defensible measurement tool.  

1. Normalize the data: Since the datasets you have for each indicator likely rely on differing units of measure, you’ll need to find a way to put all the data on the same scale so we can combine and look across datasets. This process is called normalization. This is the most math you’ll need to use in this guide. The process is broken down step-by-step later in this guide. It may sound complicated, but normalization is well within your grasp.
2. Produce an indicator score: Once you’ve normalized the data, you’ll combine the normalized datasets, sometimes with weighting and averaging, to identify a score for each indicator.
3. Produce the fully compiled indicator (Optional): By creating a normalized score for each indicator, you have successfully created a compilation from which you can understand the effectiveness of your intervention and the big picture of your work. If you would like, you can take this one step further and create a fully compiled indicator score.
   
   This final score has advantages, like ease of communication, and disadvantages, like obscuring the underlying indicators. Making the decision about whether to show your composite parts or combine into a single score is up to you and the context of your work.

Advantages of fully compiled indicators 

1. Ease of communication: Since a compiled indicator is a single number, it’s easy to communicate to both leadership and to the public. 
2. Big picture view: Through compiling, we can see our daily work as part of a much bigger picture. 
3. Lateral understanding: Because compiled indicators include not just our work but others’ work as well, we can start to understand and track how that other work affects our work, meaning we break out of silos.    

Disadvantages of fully compiled indicators  

1. Obscurity: Compiled indicators can obscure data and details, so carefully footnote, link, and provide transparency into your methodology. 
2. Sprawl: Including too many indicators will result in a composite that’s so vast you can’t responsibly track movements in it back through the indicators, and into datasets. To properly scope your work, we suggest the Rule of Eight.
3. Noise: Compiled indicators owe “...more to the craftsmanship of the modeller [sic] than to universally accepted scientific rules for encoding…” This means that you could encounter noise about the selection of datasets or indicators. To avoid this, be careful to be transparent in your methodology and craft a compiled indicator that is defensible, replicable, and verifiable. A well-crafted, sturdy indicator will stand up to any questioning.
4. Crowding: Crowding is what happens when the noise gets so loud that people splinter into competing groups. When that happens, several groups produce competing compiled indicators describing the same problem. The issue is that all of them can be right, but they’re usually slightly different in their logic and optimization, meaning none of them is the one compiled indicator to rule them all. In these situations, the only recourse is to get together and work it out. 
   
   First, remember that you all share the same problem of measurement. Then, come to the table. You may find that each compiled indicator describes different facets of the same massive problem, in which case all the measurement tools can be useful. You might find that some parts overlap, so you should work with the other team to decide how you might swap datasets in and out of the different compiled indicators to make each one stronger. You may also find that there is meaningful competition between equally strong compiled indicators, so you could consider use cases for employing both simultaneously, to develop the best solution for your big picture.