09 Aug What to do when there are gaps in data? by Julie Alsup, GPC

You want me to write about what? How can I write about progress when the right data wasn’t collected to measure progress?

Grant professionals are frequently faced with the reality of gaps in data in pre-award, and post-award. We are asked to respond to sections which require a discussion of national, regional, and local data to justify need; as well as sections requesting data-supported rationale for the proposed intervention, and finally a proposed series of measurable objectives indicated by an improvement over baseline. Sometimes there is something to work with. Oftentimes we are asked to work magic!

When I say “pre-award,” I mean to the time from the proposal creation to beginning of the grant period of performance. Gaps in data in pre-award likely means you struggled to find national, state, or local data to justify the need or to support your rationale. Or perhaps you lack baseline data that will serve as your starting point for setting and then measuring your outcomes.

If you are a consultant, before you spend a great amount of time googling and mining through websites, I encourage you to ask your client if they have any subject matter experts who may have working knowledge of the literature. Ask them if they can provide you with any data sources that are commonly accepted in the field in which you are writing. Do you know what data you wish they could provide you? If so, create the sentence prompt you are looking for and send it to your client or internal subject matter expert with a question that says “can you fill in the blanks for me?”  E.g., Between 2015-2019, the average percentage point difference between six- and four-year graduation rates of students who were eligible for Pell grants was _____. Highlight what you want them to fill in.

Here are some reliable resources for national, state, and local data sources. For additional resources check out, Building Your Data Library.

• County Health Rankings and Roadmaps
• US Census Bureau – Explore Census Data
• State departments of education (e.g., Kansas State Report Card) which provide searchable data at the state, district, and building level.
• The county or regional coalition on homelessness. You can search “(county) PIT count” or “(county) homelessness coalition” to find links to the most recent point-in-time count of the local population of people who are experiencing homelessness. (e.g., Greater Kansas City Coalition on Homelessness)
• Your local United Way

“Post-award” is the term referring to the time after the grant is awarded. Gaps in data during post-award likely means that there was a misstep or a challenge in collecting the data that the awardee stated they would collect. Without this data it is difficult to measure outcomes resulting from the grant.

As grant professionals we have learned to tell clients that if you didn’t measure it, then it didn’t happen. The reality is that staff turnover, competing priorities, lack of accountability, and gaps in communication often lead to data not being collected. Maybe pre- but not post-surveys were not administered. Perhaps the wrong data or incomplete data was collected, or important information was left off the data (e.g., participants failed to put any type of identifier code on their pre- and post-surveys making it impossible to match up pre/post surveys). This is more common in nonprofit agencies than in larger institutions which may have a post-award grants office or designated grants management professional.

Here are some simple ideas for addressing gaps in post-award data collection and measurement:

• Acknowledge the reality early and be up front with the team. As soon as you identify the problem, seek understanding about the scope of the problem. Is just one cohort of data incomplete, or is all data incomplete? As soon as you understand this, evaluate whether it can be remediated in time to salvage data collection and measurement to meet the requirements. Communicate any challenges with your funder. If you do so in time, they may even be able to help identify barriers that can be eliminated to future post-award data collection challenges.
• If data was not collected during the grant, is there anything you can do to retroactively gather the required data. For example, if the program director left and the post-award surveys were not administered on the final day can you administer an online google form survey to participants through their email addresses gathered at registration?
• Talk about what data you do have or may still be able to gather. Can you convene a focus group, or conduct some focused interviews with the participants to gather qualitative data? If the reporting process allows any space for narrative text, you can use some of this rich qualitative content to fill some of the quantitative gaps. When there are gaps in what you were supposed to measure, attempt to fill them with anything you did gather. That is a better alternative to simply leaving a field blank or just saying “we didn’t gather the required data.”
• Finally, take some time to discuss with the team and understand why the data collection didn’t happen as planned. Funders are information gatherers. They had you gather data not just to help demonstrate impact, but also for them to gain knowledge and understanding. If you learned something meaningful about why you couldn’t gather data, that should be part of the reporting process. Perhaps you found that the survey instrument that you proposed created barriers for effective utilization by the target population. Maybe it was at the wrong literacy level. Maybe the kids in the program were experiencing survey fatigue and they didn’t realize there was a second set of questions on the back page of the survey. These challenges are all part of the learning.

Standard processes and procedures for post-award grant administration can be very helpful. Ideally, the grant professional can help create these. Once the grant is awarded, have a post-award meeting with individuals who will “touch” the work including finance, operations, and program. Clearly define the following components of the methodology. Ideally this exercise also took place during pre-award before the grant was written and the post-award is simply a re-confirmation of who will do what!

For example, your desired outcome is that 80% of students in the tutoring program will improve their academic performance in math. The following breakdown of methodology may be helpful to share with your team to define roles and to mitigate any gaps at each stage.

• Data – what is the data that is of interest for this grant? (student grades)
• Data source – where does the data come from? (report cards)
• Data collection – how will you collect the data and who will do it? (students will bring their report cards to the program director at the end of each semester, or the school’s data and assessment office will provide student grades to the program using deidentified student codes on a semester basis)
• Data management – who will put the data where? (the program director will enter the grades into an Access database at the end of each semester)
• Evaluation – who will do what with the data? (either an internal or external evaluator will calculate semester to semester change in student GPA)

Discovering gaps in data is a challenging reality for all grant professionals. If you’re stuck in tough spot, we understand and have been there! Our grant professionals can help mitigate your pre-award and post-award data gaps and help you find a path forward. For more information contact Assel Grant Services today.

GPC Competency Alignment:

Competency 4 – Knowledge of how to craft, construct, and submit an effective grant application Competency 5 – Knowledge of post-award grant management practices sufficient to inform effective grant design and development. Competency 8 – Knowledge of methods and strategies that cultivate and maintain relationships between fund-seeking and recipient organizations and funders. Competency 9 – Ability to write a convincing case for funding