Episode 286: The Use of Data and Removing Bias - Known and Unknown
I'm always so appreciative when you take a few minutes of your day to join me, Randall, on this or any edition of Around with Randall. If we don't have a discussion about big Data AI, the big picture of why this is important, then I'm being now negligent in my classroom teaching activity here in this 21st century classroom.
And so we're going to delve into today some big thoughts about data and AI and really more about how organizations are using it and really what's guiding their decisions. What I really want to get at is, is that is there bias in the data that we're using and in possibly in the AI world that we all are moving into?
So we'll again start at the very top. Work our way through some of the challenges and then into some practical solutions. I want to start with a recommendation. I don't do this very often, but I will in this case is that if you're looking for a very deep, robust, more meaningful set of knowledge when it comes to AI and to data, there's really only two people that I recommend, dear friends.
Number one is Nathan Chappelle. One is. The second is Scott Rosenkranz. I recommend you follow their podcast, nonprofit AI. They have a book that they've published. The nonprofit AI. We're not going to, in 20 minutes get through that level of detail. But if you are interested in a higher level of acuity when it comes to the details into the intricacies of data and AI.
Nathan Chappelle, Scott Rosenkranz Follow them LinkedIn on their website through their podcast nonprofit AI. Really, what we want to do here is talk about the tactical ways in which we're using this. So where I'm seeing data, and particularly AI being really stretched, is the idea of identification of the possible donors that we have, maybe even segmentation of how we send out information or solicitations, some campaign strategy conversations.
I would also advocate that things like who we invite to advance maybe portfolio, a little bit of stewardship planning and certainly predictive modeling that data is driving in the emergence of AI and doing millions and millions and millions of calculations upon millions and millions and millions of variants upon millions and millions and millions of different options, variability that what we get is outcomes that we're hoping drive more philanthropy.
The problem is, is, is that this is only as good as the data you put into it. And I don't think that part is talked about enough. And that's really where we're going to focus today. That all too often we think, well, you know, I've got all this data and it'll give me the best answers and it's cruddy data and it's just going to lead you down a bad path in fundraising.
The data influencers, the data AI experts are really getting attention. And I'm not saying they shouldn't. I use AI every day, but I'm incredibly careful on who I trust because what I appreciate, and particularly with my friends Nathan and Scott, is they will tell you, I don't know what I don't know. Where I get really leery is when I hear someone say, well, I got all the answers, or our products got all the answers.
And I'm like, I know that's not factually true because the smartest people in the world don't have this figured out, that we get into a pattern where we overlook some basic things that we can control. Bias and data isn't necessarily bad intent. It comes from, more often than not, historical patterns, things that we've done in the past, or it's incomplete records or incomplete pieces of data.
It's certainly things like false wealth screening, which will spend a little more time talking about here in a few minutes. It's incorrect coding or inconsistent coding that we maybe have done over time, and its reliance on models that may not actually work. What this does is lead to bad, biased outcomes when we don't do it correctly, like missing qualified donors or telling gift officers.
Well, these are really great donors, and they're running into walls trying to actually get in front of them and to actually have the conversations we want to have it also, because we're using, more often than not, historical data, sometimes, quite often, it's reinforcing outdated assumptions that things were done in different times for different reasons. But today we may need different ways of looking at it.
It also prioritizes familiar donor segments. The more data you have on will get more into this on a certain group of people, the more likely it is for the right models to tell you these are your best prospects. But the only reason they're doing that is because there happens to be a lot of data on those people. It also under utilizes or doesn't bring about the most robust discussion on what might be considered emerging communities or constituencies.
Maybe that's new or alums or parts of your community that you didn't know or one time attendees of events. It creates blind spots. Finally, in the if when it's done incorrectly in campaign planning. And so all this is to say is, is that we've become more and more trusting of big data, of AI because we think it feels objective.
It takes out all of the parameters. And what I'm finding myself saying more often than not is, is that data really reflects the data you have and the data choices that humans have made to use it. Historical behaviors that have been decided on in the past, and also the ability or the knowledge that we have system limitations, whether that's using outside sources, which we're going to talk about or inside sources within your CRM, is, is that we have limitations on what those systems are, depending on what our CRM or our or our outside data analysis allows us.
If we don't talk about this internally, what we're going to get is really bad outcomes or less or outcomes than we want when we think we're doing all the right things on the front side. So where are these challenges? Where's this bias showing up in more detail? Well, the first and foremost is, is that I'm not saying this is bad, but I think that it's worthy of discussion or thought.
Number one is that historical bias is put on your existing data that we wait. Things like donor pass, donor behaviors or donations for future models way too heavily. If you've not had a conversation with someone who's giving you 50 or $100 every year for 15 years, then you're missing out on pieces of data about, are they interested in more?
Is there a plan giving aspects? But the data may say if we just run it from an objective in air quotes perspective, well, that person will only give $100. I get leery when you have conversations with gift officers, particularly when it comes to identifying possibly prospect management or prospect portfolio opportunities is, well, our top donors look like this.
So everybody that we want to talk to should look the same, which misses the whole aspect of why gift officers, if they're doing their jobs correctly, are so critical. They go in and they verify things and they build out what I would call qualitative data points, the relationships, not quantitative data, and run through a computer qualitative in terms of the interactions to figure out what's best for that particular donor, what they want to do.
Historical data can reinforce past fundraising habits or outcomes rather than really reveal future opportunities. The second thing is, is that most of our data collection, the big challenge is, is that it's inconsistent, uneven, incomplete. Many times I've come into organizations is a consultant, or even when I was the chief development officer and all three places that I was in my career.
Odd thing was always Chief Tillman Officer, is that I had aspirations for greater data content, data, data accuracy than my predecessors. So what happens is, is that what I was asking to be put in or directing, to be put in for portfolios or profiles or data collection was totally different than in the past. Well, how does that affect your thought process looking forward?
The other thing is, is not everybody has the same amount of data. Some profiles are very rich, others have no usable data at all. I'll give you an example. In my second chief Development Officer role. What we began to realize is that the last maybe 10 or 15 years of classes, it was a secondary educational school, private, military, all male, college prep, Catholic, a really interesting place.
The last 10 to 15 years had immense amounts of data, but the people that we really wanted to build the most meaningful relationships with, people who were probably a little older, who had more resources, at least some of them. There wasn't a lot of data at all. So all of our kind of analysis was telling us, well, these are our best prospects.
And so we went on a project and one of the project was about 18 months, maybe 16 months long, where we went back through every annual, every yearbook that we get our hands on, which goes back 125 years. And we went through every club, group sport and started really purposely populating those profiles so that we could begin to see a more complete kind of understanding of who our best donors were based on, upon what they were connected to.
This does not even take into account when we talk about uneven data collection about things like, well, screening, because some people have a lot of wealth screening data points, others have very little because they do. They're giving through a donor advised fund or their company or a spouse. All this to say is, is the more that you can do that, I would call actual live data entry things like a attendance, volunteer records, family relations.
While if you only have a certain number of those, it's going to create an inconsistency in the analysis. The more you have that, the better off you are. Some donors will feel if you don't do this correctly that appear to be stronger prospects, possible donors simply because they have more data attached to them. The third is, and this is you've heard me say this multiple times.
I'll mention it. Here is our third of five kind of challenges and then kind of move on. But is is that we have a reliance on predictive modeling that's in, well, screening. That's just an assumption by gift officers that it's absolutely the go to for anything, that it's always correct. I'm like it at best. It gives us a directional viewpoint of what might be possible with someone.
It doesn't replace the qualitative relationship building process that we should demand of all of our gift officers. I'll move off that. You can go back and listen to about ten other podcasts where I spend enough time talking about that on my own particular perspectives on it. The fourth is, is that when we talk about some of the data points that come in, there's some what I would call proxy problems, i.e. what happens when the zip codes are wrong or we don't have real estate ownership correctly, or giving history.
Let me give you an example. A number of times I've run into this both as a practitioner and as a consultant, where somebody thinks, oh my gosh, look at this as a holy grail of donor opportunities. But they actually live in a trailer, a mobile home. Well, the reason that came up was, is that they're just renting that particular space, and it's taking into account a large mobile home park which they don't actually own, or an apartment.
Same thing. And so we have these proxy problems because we assume the data we're getting maybe from outside sources is accurate. These tell us only a part of the story. You have to go verify these things, but they don't create the whole picture. The last is, is that no matter how much we think it is, that data is objective, it's colored by human bias and the use of the data.
Even clean, data rich meaningful data can be looked at in ways that create, through the human eye, a sense of bias.
Think about that. We ignore things like anomalies, or we make assumptions based on familiarity, or we think we know something that we actually don't. We build portfolios at times based on convenience rather than on opportunity because we, you know, oh, this is a good doctor. If it's a health care, I'm going to work with this doctor, all of his patients or wealthy.
Well, how the heck do you know that? These decisions, you may think, well, that doesn't happen. It happens all the time. The point is, is that technology doesn't eliminate human bias. In my opinion, it actually amplifies it. Because if you make that assumption, that analysis, that technology, the AI or whatever, if you don't do it purely is going to exponentially increase that bias that you made at the beginning.
So as we try to do each and every episode, what are some tactical solutions to try to overcome this? How can you as an organization have discussions and do a certain couple of things that might very easily help you get through some of these challenges? So number one six things is to audit your data inputs, not your outputs.
This one I, I can't believe we don't talk about more often. We're always analyzing the outputs of our data. Who's in our portfolio. What who's the best prospect. What's our best opportunities in a campaign? How would we assess campaign dollar figures in terms of what's possible? The point is, is that we look at the outputs. But how many times has anybody ever said, hey, let's look at what we have and what we're putting into the data analysis that's forming the outcomes.
Where's the data coming from? What's missing? Who had input on this on a maybe a larger scale? And what were their perspectives when they did this? Who's making the decisions on what data should or shouldn't be in there? Data audits should be about fairness and completeness, not just cleanliness. We talk about cleanliness in our database often, but what we don't talk about is what data are we using fairness.
And is it complete to be able to figure out if we have all of this data? Are we using it accurately? Is it providing us the best outcomes? We don't look at the inputs often enough as to what's driving those outcomes. So the first thing is ask questions. What data are we using and what are we using. The second is, is that data should be one input point, not a final decision.
This is the well screening issue. Predictive modeling. Well we these people should be able to give because we've run the analysis. Again that depends on about a million factors. Even AI if the data is bad can't overcome. What it does is it gives us a sense of direction. I always talk about, well, screening is giving us like we can go that way, but that is not dictating what the final answers are going to be.
We need to encourage gift officers. We need to encourage relationship builders to combine the data with relationship intelligence and volunteer engagement and personal observations. The qualitative relationship building process that really gets to philanthropic motives of our donors. I quit this one. If you follow sports to The Incredible Move and Moneyball with Brad Pitt kind of highlighted it as a movie talking about Billy Beane and the Oakland at the time.
Now somewhere else, I don't even know what they call themselves, but the A's talking about using cyber metrics. Bill James that they there's this pressing kind of give and take between data. Now, as all pretty much sports teams and even college teams now are using data to make decisions. But if all you that doesn't take away the human component of decision making, and we can't do that in philanthropy either, we need humans to be involved in the qualitative assessment of the moment that AI or predictive modeling can't see.
Third is review, portfolio and prospect alignments. Assignments in looking for patterns. Is there a concentration of certain kinds of people, donors going into certain portfolios? How do we decide who gets assigned? Who gets to choose where they're assigned, which prospects are or groups of prospects are repeatedly prioritized? Why are their entire segments that we just don't even touch?
We're not even doing outreach to? I'm not sure I have the answers to any of these questions that I just gave you in this kind of reviewing the process of prospect and portfolio assignment. But what it does is if you ask the questions in a meeting and a conversation reviewing process, you actually might find out that you're doing some things that actually are hindering your gift officers from meeting with the right and trying to get Ahold of and trying to build relationships with the right people.
Number four is improve the data capture during the donor journey. When I'm when, when your gift officers, when you are a gift officer and you're in the relationship building process, then we should be using our attributes and our constituencies much more robustly when we learn things about donors, where they went to college, where they grew up, what else do they give to?
Records are more than transactions capturing things like volunteer engagement, event interaction, stewardship responses, affinity indicators, family relations, behavioral signs, even into the nebulous world of social media, which I don't want that much about. What are they saying? How are they saying it? All of those things shape data outcomes as maybe enhancing them because they're better data inputs, better inputs produce better analysis.
But yet some of the most richly needed data comes from the relationship building process. And we get action reports and say, I met with Bob and we're going to meet again. What about talk about what was important and then all of the things that did they show up to event. How did they respond? Did they do a survey?
The more of this kind of depth gives you better outcomes. Number five build some type of cross-functional review on. I'll call it a regular basis. I'm not saying every month, two months, even every year. That's depending on size and scope. But what am I talking about here? I'm talking about letting the people who control the data do the data analysis of the data analysis.
You want to bring outside people, and they don't have to be outside vendors or consultants. They can be, but you want somebody else's eyes on this. I equate this to the way I look at taxes. When you are a complicated filing of taxes, you're 1040. Maybe it's your complicated 990 for your nonprofit. It could be corporate taxes. Who knows when their complicated accounting will have multiple levels of people checking and looking at the data, looking at the tax return, and they want different size on it.
So make sure nothing was missed. The same applies here when you allow the same people to review it, who created it? What you're going to get is built in bias one way or another. So bringing in fundraisers and challenging status quo other researchers, maybe it's maybe it's other members of the database staff, maybe it's the leadership. Maybe it's a consultant.
Multiple perspectives will help you reduce blind spots so that it actually allows you to be more effective with the data analysis you're doing. Number six, don't be afraid to change the model. I'm not saying to it every month because that's not helpful. It's kind of productive and counterintuitive. But who are we consistently missing? That's an interesting question.
But the more interesting question is how would we ever find that out? And that's what we need outside people sometimes to come and look at, maybe challenge us is to say, who are the people that are giving unexpectedly? And who else might be someone like them? Where did that data come from? What was that analysis like? What assumptions in the past have proven wrong?
Which ones are proven right? What prospects succeeded outside of our modeling principles, meaning they performed at a higher level? Why learn from these exceptions learned from these deviations? When we talk about statistics and data, don't be afraid to challenge and change the modeling process, at least a view of it. Because if you don't, I guarantee you you're just going to miss people.
It's just innately a part of that. Six things that you can do right now. Audit it. Make sure that you are not using just one kind of input. You're using multiple levels of input and most importantly, not allowing to drive the final decisions that that data input, that there should be some humanistic pieces to this, that you want to review the kind of the assignment patterns and see if there's something missing.
You want to look and see. Are there more data that we can get in the qualitative pieces? The Relationship Building Process Gift officers reports that we want to use some type of analytics to kind of what I would call cross-functional review the different kind of data points to see who else might have other data points, to see if we're missing something that might be valuable.
And finally, don't be afraid to challenge or change the model a little bit to look for things that might be the exceptions. That maybe should be more of the rule. Data driven fundraising is incredibly powerful. It's the future. I'm not advocating going back to pen and pencil on all this, but we don't talk enough about fairness, completeness, thoughtful interpretation of what matters in this overall discussion that those things matter almost as much, if not more, than the analytics.
The goal isn't perfect data or outcomes. The goal is better decisions or directional decisions that have a broad vision and as little or few blind spots as possible. And if we do that, what we get is the use of data to sharpen either the identification of the best options maybe heightened, sharper, and opportunities for better relationships. It's not to narrow only the people that we want to talk to be mailing to.
It's to create better options for our outreach. The bottom line is, is that if you're not questioning some of the things we're talking about here today, then you're going to have a lot of blind spots and a lot of data analysis, AI, whatever you want to call it, that is not providing the best options for your organization to raise money.
And that should be at the top of everyone's list because these nonprofits, your nonprofits, the nonprofits everywhere need this money for investment to make the world that they live in, that they operate in, that their mission drives them to a better place. Don't forget to check out the blogs and Hallett Philanthropy two per week and feed. Really kind of always surprised more and more so how many people are reading them?
90-second reads at Hallettphilanthropy.com/blogs. Or click on that hyperlink and create a feed. Your work what goes on every day is critical. It's not just about your value, but the value that the nonprofit that you work with or volunteer at or lead or whatever are making a difference in the community for people, for people's experience or outcomes or direction or whatever it might be to make the world just a little bit better.
And in doing so, what you do brings great value. It's critical. Don't forget my all-time favorite saying some people make things happen. Some people watch things happen. Then there are those who wonder what happened. You're someone who's making things happen. If you're doing the work that is on your desk or in your on your to do list every day.
Building relationships, supporting those who build relationships, making a difference in the organization. Who has this mission to make the world just in your area of expertise a better place. And in doing so, you're changing lives. For those who are wondering what happened to many times, there's too many people, too many places in our community that are wondering what happened, what a great way to spend a career knowing that you are helping them value getting up every morning saying, yeah, what I got to do is pretty hard and I'm not paid enough.
But I'm doing positive things. That's worth it every single day. I look forward to seeing you next time, right back here on the next edition of Around with Randall. Don't forget. Make it a great day.