What should I monitor?

Looking at all the initiatives or activities your section is planning or implementing and considering the notion of monitoring all of them might be cause for alarm. The good news is that you are not expected to monitor all your activities or initiatives.

Monitoring selection criteria

PD sections cannot monitor everything they do, nor would doing so be a good use of a section’s time. Choose what your section will monitor based on: (1) whether something is a relatively resource intensive activity; (2) how closely it aligns with your team’s strategic priorities; (3), whether you anticipate it being subject to a data call; and (4) whether it is already subject to other monitoring requirements (e.g. those funded by bureaus such as ECA or DRL).

We recommend that PD Sections use the following monitoring selection criteria to identify where to prioritize monitoring efforts. These criteria include questions about the section activity itself, as well as whether section activities have alternative monitoring requirements (such as from ECA or DRL).

1- Questions about the section activity itself:

• Does this section activity require a large outlay of resources? Instead of prescribing a fixed percentage of the budget to guide monitoring decisions, this guidance encourages PD sections to prioritize monitoring for activities that use more than the average amount of resources (in terms of staff time or money). If it is not a resource-heavy activity, you likely do not need to complete the full PD monitoring process, instead focus on collecting basic counts or outputs of activities to understand what happened.
• Is the activity connected to your team’s strategic priorities OR do you anticipate reporting on this activity in a data call? Strategic priorities may include emerging foreign policy priorities, such as those reflected in the NSS, JSP, or JRS, or a priority reflected in your ICS or PDIP. Alternatively, you may anticipate a data call as the result of a White House or 7th Floor speech or special initiative. If the answer to either of these questions is yes, even if the initiative/section activity is not resource-heavy, you may consider completing the PD monitoring process.

2- Questions about whether the activity has alternative monitoring requirements:

• Does this activity have alternative monitoring requirements dictated or controlled by the funding bureau or office (e.g. ECA)? If not, you should complete the PD monitoring process.
• If the activity DOES HAVE alternative monitoring requirements, does the data currently collected tell you what you need to know? If it does, do not complete the monitoring process. If the data does not tell you what you need to know, either a) work with designers of the existing framework to add elements to collect the information you need or, b) if this is not possible, complete the PD monitoring process.

Monitoring for initiatives, section activities, or both?

Throughout this document, we refer to “section activity” monitoring, which conforms to the PDIP process and PD Tools nomenclature. Some PD practitioners refer to these as programs.¹

Initiatives are collections of related section activities that work together toward achieving an ICS sub-objective. Section activities are activities, events, projects, and digital or traditional media campaigns that contribute directly to achieving goals of their parent initiative.

Although 18 FAM 300 allows for monitoring at either the program (initiative) or project (section activity) level, this document encourages sections to focus on monitoring at the section activity level.² The section activity level is the level that contains a defined intervention, tailored to a defined audience, and with a series of expected outcomes. Monitoring initiatives may be an appropriate approach when the associated section activities share a single audience segment and set of outputs and outcomes. PD sections should contact R/PPR/REU for support in developing outcomes and crafting the monitoring plan for initiatives.

Conducting monitoring at the initiative level is most useful with initiatives that engage with the same individuals over a period of time, and where each activity attempts to produce and/or reinforce the same outcomes. If your initiative activities share the same participants and the same outcomes, you can use the same tools for data collection. Keep in mind, though, the “same” means identical, not “similar.” Although it is not always the case that each activity engages the same individuals (participants), it helps when it comes to data collection.

Conducting monitoring at the section activity level is most useful when each activity engages with different individuals over a period of time, and each section activity has different outcomes. In this case, the initiative is important for refining the theme of the activities, but it is not the source of your outcomes. Practically speaking, you might look at an initiative and identify only one activity that qualifies as a good candidate for monitoring. You then engage in the monitoring process outlined in this toolkit just for that particular activity.

See Table 1 for an example of monitoring at the initiative and section activity levels.

The Emerging Entrepreneurs Initiative in Table 1 is a good candidate for initiative-level monitoring because each nested activity includes the same audience and seeks to achieve/reinforce the same outcomes. This consistency provides a good opportunity to collect monitoring data on the initiative as a whole, rather than each individual activity within the initiative.

Conversely, the NATO Outreach and Security Cooperation would be a good candidate for section activity monitoring because each section activity has different outcomes and audiences. If you were to create broad outcomes and collect the same type of monitoring data for each activity, you risk not measuring the right thing.