BI

Letting Go of Sacred Cows: The Danger of Being Data-Driven

So you want to become more data driven, right? Everyone does. Everyone is sold on the power of analytics to enable smart, data-driven decision making to improve organizational results. I don’t think it’s entirely unfair to say that terms like “big data,” “analytics,” and “data-driven” are so deeply ingrained by now as to approach buzzword status (not a good, thing, of course!).

But are you really ready to be data-driven, to go wherever the data leads you? It’s often not as easy as you might think it should be for people and organizations to allow data to guide their decisions, tactics, and strategy, especially when the data contradict something they just “know” to be true. Aren’t some things simply given? Simply and obviously true a priori? We all hold these kinds of “convictions” about something, and when we encounter data that contradict what we “innately” know we naturally resist and find ways to rationalize the data away.

There was a perception at an independent school for which I worked (in advancement) that the school’s culture and programs were not adequately “girl-friendly,” which was driving increasing attrition among female students, particularly at the key transition point between grades 8 and 9 (it was a K-12 institution). There was considerable speculation, particularly at the Board level, about what needed to be done to fix the problem. Not atypically, there were some loud voices on the board seeking to push various pet programs, which their advocates intuitively “knew” would improve results.

It happened that we were working on an institutional research project for admissions, and we had enough data to take a look at gender and attrition. We needed to quantify the problem of accelerating female attrition before we could look for correlations. What we found was that there was no particular female attrition problem. In fact, the retention rate for female students was slightly higher than that for our male counterparts. What was really happening was that way back at the start of the pipeline, in grades K, 1, and 2, the balance of students was 60% male, and that balance largely held all the way through. Not exactly something that should have been hard to notice or uncover, but the perceptions were established and this was overlooked.

Did this “revelation” put an end to the calls to add a cheerleading program to the upper school (one board member’s solution for making the school a better place for girls)? The title of this piece provides a hint. The certainty about the female attrition problem survived unscathed. We would like to think we are fundamentally rational beings, but it is hard to let go of closely-held beliefs, no matter how information points in another direction.

What does this kind of resistance to what the data are telling us look like in advancement? It takes many forms. Here are a few I’ve seen recently:

  1. Continuing to mail the same number of solicitations annually to a large pool of never-givers, despite models that suggest de-emphasizing a population and re-directing resources to more likely constituents.
  2. Spending significant gift officer time on statistically low likelihood prospects who have extremely high wealth ratings (yes, it’s fine to make a discovery call if the prospect will take an appointment; but at some point it’s time to move on).
  3. Allocating significant personnel time to maintaining some sort of “extremely important” constituent list, despite not knowing whether there is any correlation with giving (and sometimes, even knowing that no such correlation has been found).

Some people have very little difficulty accepting data-driven conclusions that run counter to accepted wisdom; in fact, some people enjoy the power of data to undermine unexamined assumptions. I know I find it quite exciting. But I’m sure if some Sabermetrician showed me advanced statistics indicating that up to this point in his career Novak Djokovic has been a better player than Roger Federer during a similar interval, my first reaction would be something along the lines of “that’s not what I’ve been watching.” But, as a data-driven fundraising practitioner and consultant, I would have to shrug that off quickly and listen to what the data have to say. In the end, our biases can be overcome, but it does take a degree of conscious effort and repetition. If you’ve made a commitment to becoming a more data-driven fundraising organization, you’ve taken the first step. Remind yourself to hold opinions lightly and to be open to the messages in your data.

Brandon Ferris
Senior Director of Strategic Services and Fundraising Counsel
Zuri Group
http://www.zurigroup.com
@datadrivenbf
brandon@zurigroup.com

Advertisement

In Defense of Crystal Reports

I encounter quite a bit of bashing of Crystal Reports within the advancement industry. The criticism seems to be growing of late as more people within the industry gain some level of awareness of competing reporting tools such as SSRS, Tableau, and Logi Analytics, to name a few. These are powerful tools, to be sure, and they may each offer some unique capabilities. But Crystal Reports is also extremely powerful if you are willing to invest the necessary time and effort into learning how to leverage its capabilities.

Which brings uCrystal_Screenshotp an important point: All reporting tools require a significant degree of training and understanding. Let’s be blunt for a moment: Many of the reports we want or that management requires are highly complex, and they are reflective of highly complex fundraising operations and organizational structures. You can only go so far in terms of simplifying the process of generating the needed analytical reports. It will never be “easy button” level easy. So before we dismiss Crystal Reports as being “too complicated,” let’s step back for a moment and admit that this whole endeavor is highly complex! And no matter what tool you use to create reports, you have to have a solid understanding of the underlying data. Selecting the data you need to answer your reporting question is more than half the battle.

One of the great advantages of Crystal Reports is there is a high likelihood you already own it, either because it’s so ubiquitous in the industry or because it was bundled along with your fundraising database system. Tens of thousands of organizations using The Raiser’s Edge already have Crystal Reports, even if they’re not using it or are not using it extensively. And RE even has a built-in export format designed for Crystal Reports. When you combine that with a user-friendly query tool, the process of pulling the data and joining the tables you need for a report is made relatively easy. If you use a different CRM system, Crystal still makes it fairly easy to connect to the database to select datasets directly, or to use datasets in various formats.

It ultimately comes down to training. I’m not sure I have ever encountered an organization that has provided enough training for its employees. In all my years as a fundraiser I received exactly ZERO training to use my CRM and reporting tools. Like many of you out there I had to figure it out on my own. That’s great once you get there, since you’re quite likely to retain that hard-won knowledge, but it’s a terribly inefficient way to go.

Well-designed training can get users up and running and generating great Crystal Reports in a matter of days. Of all of the various training topics I deliver I enjoy teaching Crystal Reports the most – by far. It will sound like a cliché, but the most rewarding thing for a teacher is to watch the light bulbs go on for their students. When my students discover that they can understand Crystal Reports, the enthusiasm they exude is infectious. I hear exclamations like “I can’t wait to use this” or “I’m going to be able to do so many things with this!” all the time.

Brandon Ferris, Senior Director of Strategic Services and Fundraising Counsel
Zuri Group
brandon@zurigroup.com
@datadrivenbf