Finding Signal Through The Noise
Identifying Health and Performance Metrics That Work For You
Much of my work these days revolves around helping my clients and students develop a more holistic understanding of performance longevity metrics. I argue that the purpose of collecting data is to help us calibrate our perception. To better understand our subjective experience so we can mitigate the costs of performance and life for that matter.
If we can do so that we can become attuned to signals whether from external data (outsourcing) or internal information (insourcing) so that we can take informed action. m.c. schraefel (it's not bad grammar-it's lower case) and her colleague and co-author of multiple papers, Erik Hekler (a professor at UC San Diego) describe the insourcing and outsourcing continuum in their explanation of tuning, an idea of continuous adjustment to your health needs and demands over time based on informed experience. I wrote an article about it HERE.
Their ideas have inspired me quite a lot in the last 18 months and how I think about more effective ways to keep people engaged with the gathering of information to calibrate, attune, and tune themselves. One of the challenges I bump into often is the shear volume of options when it comes to measuring health and performance. All competing for your attention in the open marketplace of ideas.
If you don't have expertise in the field of health it can be quite a task to identify which gadgets and reports you should listen to and which ones you shouldn't. To that end I'd like to share a portion of the overall Check Engine Light curriculum that I think might be helpful.
In class we build a Performance Longevity Dashboard, a place where all of the indicators you might use will be collected. This could be a real physical place or a file in your head. When identifying indicators you might choose to help you pilot your health and performance more effectively I recommend asking the following:
Is it valid?
Is it reliable?
Is it accessible?
In my experience this can help inform whether or not the indicator(s) will sustain a relevant and meaningful place in your understanding of your own experience of body and mind. It can also help to protect you from putting too much weight on outsourced metrics which are often not as accurate as you might think.
Let's look at each in more detail.
Valid
Indicator validity means that an indicator accurately measures that which it is intended to measure. For example, devices that measure heart rate using infrared sensors on extremities are notoriously inaccurate during physical exercise. That's because they use LED sensors to measure pulse rate which increases in blood pressure during exertion render terribly invalid. This type of validity specifically is referred to as construct validity. Is the measurement a legitimate reflection of thing being measured?
One way to check validity is of a metric device is to see if it's been measured against a gold standard (this is referred to as criterion validity). For example, has your watch's heart rate monitor been tested against a chest strap that measures electrical activity directly from the heart? If not it doesn't mean you need to trash it. But be careful how much stock you put into decisions you make based on that information alone.
A couple of other points to consider when determining if your indicators are valid is if they accurately reflect the totality of the situation (they usually don't). Is your HRV score alone really enough to tell you how you should approach your day or is more information required? Lastly, and while less scientific is equally important is does this indicator make intuitive sense as a reflection of what you're trying to understand? If it doesn't the likelihood that you'll change meaningful behavior is low.
How to check validity:
Has the device or method been tested against a gold standard? (e.g., a chest strap HR monitor vs. ECG data)
Does it truly reflect the aspect of health or performance you want to measure?
Does it make intuitive sense as a useful metric? If not, you’re unlikely to act on it in a meaningful way.
Reliable
Validity is an important consideration when building a dashboard of indicators but it's certainly not the only one. Reliability matters too. In this case reliability means that an indicator produces consistent standards of measurement over time under stable conditions. If I'm wearing an OÅ«ra Ring to bed every night I want be certain that the sensors maintain reliability even as the battery loses charge. As an anecdote one thing I've noticed with the OÅ«ra Ring 3, the one I currently wear, is that the LEDs are significantly and more consistently brighter than the prior model. To me, that speaks to both validity and reliability-at least at face value.
Indicators we use can be reliable but not valid. If you get a scale that measures body fat percentage from Target that measurement may be consistently executed by the device but it's not nearly as accurate as a DEXA scan. Again, that's not to say to throw the baby out with the bath water. You may be able to get some sense of trends over time but you can't step on one of those fifty dollar scales and take the results to the bank.
Additionally, we can have validity but not reliability. You might be using the heralded Omega Wave to measure your HRV every day but if you put the sensor pads on a sweaty body or don't place the electrodes properly the measurements you get will lose some reliability. This is where having a solid standard operating procedure can help. (Men: read the instructions.)
Key reliability checks:
Does the same test produce similar results under stable conditions?
Is it resistant to environmental factors like placement errors or battery fluctuations?
Some tools are reliable but not valid (e.g., a cheap body fat scale that gives consistent but inaccurate readings). Others are valid but not reliable (e.g., a high-end HRV monitor that gives inconsistent results if electrodes aren’t placed correctly).
Pro Tip: Follow a standard procedure when measuring things like HRV, weight, or resting heart rate to improve reliability.
Accessible
The third sibling in this little family is accessibility. Accessibility is often where the rubber meets the road with real world applications of using indicators to help us calibrate our perception. If you don't have access to something however scientifically valid or reliable it might be means jack shit.
In my view accessibility has two components to it that make important differences when it comes to what you're going to put on your dashboard. First, physical proximity. Can you get to it? Do you have it? With enough regularity that it's a reasonable way for you to assess or monitor yourself? This one is kind of obvious. If I can't get to a thing, it's not accessible.
The second part of accessibility is not quite as obvious. In order for an indicator to be truly accessible you have to understand the information it is giving you. If it's just numbers and science mumbo jumbo that is no help. It does not matter if you are wearing the latest and greatest fitness technology if you don't understand how to take meaningful action based on the information.
There are lots of runners out there with insanely sophisticated computers that they wear on their wrist. Many of those use the clock, the calendar, and the stopwatch because the rest of the functions might as well be in hieroglyphics. That's not accessible.
Physical Accessibility:
Do you have it?
Cognitive Accessibility:
Do you understand the data enough to take meaningful action?
If a metric might as well be written in hieroglyphics, it’s not truly accessible.
Example: Many runners wear advanced GPS watches but only use the clock and stopwatch—because the rest of the data is overwhelming or irrelevant to them.
Three Peas In A Pod
None of these three factors—validity, reliability, or accessibility—is more important than the others.
A cheap scale may lack the accuracy of a DEXA scan, but it lets you track trends over time.
A lab-based VOâ‚‚ max test is highly valid but not practical for regular monitoring.
An HRV score is useful, but only if you understand how to interpret and apply it.
The goal isn’t to find perfect data—it’s to develop a system that helps you calibrate your perception and make informed decisions.
Final Thoughts: The Only Constant is Change
Technology will evolve. Your needs and experiences will shift. What works today may not work tomorrow.
The key is to continuously refine your Performance Longevity Dashboard, filtering out what doesn’t serve you and keeping what does.
At the end of the day, the best indicators are the ones that help you take meaningful action—because data is only as valuable as what you do with it.
What’s Next?
Drop a comment:
What indicators do you currently track?
Are they valid, reliable, and accessible??
Share this post: If you found this useful, pass it along to someone who might need it.? Subscribe for more: Get regular insights on performance longevity, health metrics, and practical tools you can apply today.
I've been using my Garmin for tracking/trending resting heart rate and HRV and less often heart rate recovery, average HR, Vo2 max, and max heart rate . Once in a while I put on the chest strap to get a sense of what the differences are between the wrist and strap. As part of an annual cardiology work-up, I get a CPET which gives me a more accurate measure of max heart rate, threshold, HRH so I have that data as context for the data I'm getting from the watch. I weigh myself a couple days a week. I use all of this for trending and for paying attention. For example, if I find HRV or resting heart rate creeping, I may pay more attention to what is and has been going on in recent days. I've learned a lot about what is good for me and what isn't (sleep, booze, stress, pushing myself physically (eg building a deck), etc.) From this learning I can be more intentional about how I live my live. A couple of times per year I may look at 1 year or multi-year trends. I'm a 70 year old still doing a couple of half-marathons and a marathon a year after running ultras into my mid-60's. My goal is to work to remain healthy and fit within the uncontrollable context I'm dealt so that I have as many options open to me as possible. My wife and I share this goal and make good accountability partners. I'm on the downward trend as far as speed and interest in my performance numbers. I do find that paying attention to these other data trends is helpful for me in terms of maintaining my intention. Intention directs attention and attention directs actions. Rob, I find your work to very helpful and it aligns well with my objectives. Thanks for doing what you do.
Very useful information and serves as a building block for monitoring your health. I include a diary that I use to track all of the food that I consume. I include everything from the smallest item to my major meals. When I compare it on a daily basis, I can see the impact of what I eat has on my weight, blood pressure and overall feeling of being. I also use Fitbit and a scale to track my activity levels. I use a blood pressure monitor in the morning to measure my blood pressure.
The most important part of it is to capture the data in writing so that you understand what your results are.
I have found it amazing as to how little I knew about the food that I eat and what its effects are on my body.
Using your methodology, it’s like putting cheep gas in your car and wondering why you aren’t getting the mileage or performance benefits that you want.