Experiments in Closed Loop Measurement

AcxiomFebruary 05, 2016

The process of closed loop measurement could be described as something like this: We seek a specific outcome, so we measure that outcome to set a starting point, to know where we are now. Then we examine what input might impact the outcome, we make a change to the input, and give the change a chance to work. We measure again. If the outcome is improved, we might make more changes to that same input – we might optimize – or we might start looking at different, or multiple inputs. By closing the loop, swinging results back around to influence and guide our actions, we improve outcomes. Measuring alone does not improve results. Closing the loop on measurement is the key, and using an iterative approach becomes more critical as the number of inputs increases and becomes more complicated.

Weekdays, I arise at 5 am and essentially promote closed loop measurement. “What a weirdo…” you might be thinking. You see, my hobby is working as a group fitness instructor and 20-30 people join me at this fine hour of the day. While they think they are coming for exercise, they are actually an experiment in closed-loop measurement. They want to be more fit. They seek specific outcomes related to wellness. For those who just show up and hope for the best, well, I hope the best for them too. But for those who will set a baseline, measure, optimize and continue to iterate what they do to achieve those desired outcomes… those are the people who tend to realize their goals.

Consider one of my morning crowd. It’s a story many may be familiar with. “Claudia” has been coming to class for years. She has wanted to lose 20 pounds for years. (A sidebar here: the job of a group fitness instructor is to lead a group of people in some form of exercise. You do not cross the line to offering 1:1 advice or opinions unless asked.) There is anonymity in the group, which is what some people seek. However, when Claudia stated for the 943rd time “I need to lose 20 pounds” and then added “what should I do?”, this was my invitation to suggest she set a baseline. “Oh, I’m not going to weigh myself, I don’t want to know how much I weigh.” This caused me to pause. I then asked how she knows she needs to lose 20 pounds and, furthermore, how she will know when she achieves her goal. I suggested she consider some other measurable course of action, but she continues to show up and hope for the best.

Another early morning soul, “Bob”, was also attending class day after day after day. A big man (6’7” and 350 pounds), day after day after day he remained at 350 pounds. His weight was too high for him to really enjoy life and he was medicating for high cholesterol and high blood pressure. But he was doing the right thing. He was measuring. What he wasn’t doing was closing the loop on that measurement and optimizing results. But this time, when he asked “how can you help me?”, here’s what we did. Exercise certainly wasn’t hurting anything, but he was attributing weight loss solely to one input – Exercise. When we closed the measurement loop, it became apparent that other actions needed to be taken. We sat down with a food diary and examined those inputs. Bob was eating a healthy diet. There were no soft drinks, no Twinkies, no chips, no ice cream. We dug deeper and found that for breakfast, Bob was eating oatmeal and was also having a glass of skim milk (no problem there). Then, because he’d exercised before breakfast, he was thirsty. So he’d have another glass of skim milk, and then maybe one more, and maybe a couple more glasses for lunch or dinner. And he uses a 20 ounce glass. Bob was drinking up to 1,000 calories a day in skim milk! For his twice daily snack, Bob was grabbing a handful of nuts or maybe a handful of dried fruit (all good). However, at 6’7”, Bob has very large hands. A “handful” of nuts or dried fruit was adding up to another 1,000 calories per snack. The point to the story: Bob had a known starting point of 350 pounds and had a goal to see that number go down. We examined inputs. Input #1 was Exercise, and then another input, Diet. As we measured those inputs we found areas where we could make changes that should influence outcomes. Bob replaced all but one of his glasses of milk with water. He took a measured amount of fruit and nuts. He measured, he examined inputs, made changes, measured again, and iterated. He closed the loop and dropped 65 pounds. AND kept the weight off. AND no longer medicates for high cholesterol and high blood pressure. And MOST IMPORTANTLY Bob contributed a good lesson for those of us who spend our days focused on measuring marketing efforts.

Back at the office, talking with one of our data scientists here at Acxiom about the topic of measurement and specifically about attribution, his advice echoes what Bob has proven works.  Measure, examine inputs, make changes, measure again…  and iterate your way to good outcomes. Want to talk about closed loop measurement? The iterative nature of accurate attribution?  I’ll be happy to connect you with data scientists who can help.  And they won’t make you do a single squat, push-up or crunch.