Built from a Master's Thesis: The Science Behind PuffinVBT

By Gunnar S. Knudsen · · 10 min read

Built from a Master's Thesis: The Science Behind PuffinVBT

PuffinVBT didn’t start as a product idea. It started as a research question: can a phone camera extract barbell velocity accurately enough to be genuinely useful for training, without any hardware sensor?

The answer is coming from a master’s thesis in data science, still in progress. Nearly 1,700 training videos between the core validation set and real app usage. A tethered linear position transducer (RepOne) as the reference standard. A systematic evaluation across thousands of pipeline configurations. The product is the engineering output of that research, and the research itself is still running: the pipeline gets swept, tuned, and re-validated on a rolling basis, so the numbers below are current-best, not final.

This post is about the science behind the tool. What’s being studied, how it’s being validated, and what the results mean for the numbers you see in the app.

The Research Questions

The thesis investigates two questions:

RQ1: How accurately can computer vision measure barbell velocity and range of motion compared to a tethered sensor?

This is the foundational question. If camera-based measurement isn’t accurate enough, nothing else matters. The thesis measures this by analyzing training videos with the pipeline and comparing the output to simultaneous recordings from a RepOne linear position transducer, a commercial device that measures displacement directly at high frequency.

RQ2: Can VBT metrics derived from video predict an athlete’s self-reported RPE?

This is the research extension. If velocity data from a phone can predict how hard a set felt to the athlete, it opens the door to automated fatigue monitoring and autoregulation suggestions.

Where the Research Stands Today

RQ1 and RQ2 are at different stages.

RQ1 has real numbers, from the latest full validation pass across the pipeline’s method grid, run on real training footage against the RepOne reference. On the supported filming angles (side and diagonal, see the limitations below), the current best-performing configuration reaches:

  • Rep-count accuracy: about 98%
  • Peak velocity: ICC(2,1) = 0.923, MAE = 0.046 m/s
  • Mean concentric velocity: ICC(2,1) = 0.927, MAE = 0.028 m/s

For context, VBT validation studies commonly treat ICC above 0.90 with MAE under 0.05 m/s as excellent agreement. This pass clears that bar. It doesn’t mean the numbers are done moving: every pipeline sweep re-runs this same evaluation, and the config that wins changes as methods improve.

RQ2 is still early. The candidate predictors, RPE-scale mapping from %1RM, velocity-threshold approaches, and within-set velocity-loss (%-dropoff) as a fatigue signal, are all being validated against the same real training data. No results are being published yet because none are ready to stand behind. When there’s a real signal to report, this post gets updated.

How the Pipeline Works

The pipeline that processes your video has six stages, each the result of extensive research and optimization:

1. Preprocessing. The video is analyzed for phone rotation metadata and frame timing. Smartphones record variable-frame-rate video, which means the time between frames is not perfectly constant. The pipeline accounts for this to avoid introducing velocity bias.

2. Segmentation. A custom-trained machine learning model identifies the weight plate in each frame. The model was trained specifically on powerlifting footage to handle different plate colours, gym lighting, partial occlusions, and various camera angles.

3. Ellipse fitting. The detected plate contour is fitted with a parametric ellipse. A circular plate viewed from an angle appears as an ellipse, and the shape of that ellipse encodes perspective information. The thesis evaluated multiple fitting algorithms and found that the right choice of method significantly affects downstream accuracy.

4. Timeseries cleansing. The raw frame-by-frame position data is noisy (tracking jitter, occasional frame drops, slight segmentation variations). Signal processing cleans the trajectory: re-identification across tracking gaps, outlier removal, gap imputation, and smoothing.

5. Perspective transformation. Pixel movements are converted to real-world metres using the known diameter of a competition weight plate (450 mm) and the ellipse geometry. This is the step that turns “the bar moved 30 pixels” into “the bar moved 12 centimetres.” Two transforms were tested here: affine, and a full homographic transform capable of reconstructing the entire 3D plane the plate sits on, not just a flat scale correction. Homography is the more powerful tool when there’s enough geometry to constrain it, multiple tracked plates in frame can supply that. With a single plate, the common case, there isn’t enough independent geometric information to solve for it properly, and the math degrades rather than helps. Affine, tested and validated empirically against the alternative rather than assumed, is what ships.

6. Rep detection and KPI extraction. Individual reps are identified from the processed position signal. Per-rep metrics are computed: peak velocity, mean concentric velocity, ROM, concentric/eccentric phase split, and derived values like velocity loss and estimated 1RM.

Each stage was evaluated independently and in combination. The thesis tests a large grid of method choices across these stages (different segmentation models, different fitting algorithms, different imputation methods, different smoothing techniques) to find the combination that produces the most accurate results, and that grid gets re-run as new methods are added.

One more thing worth knowing: when more than one plate is visible and confidently tracked in a frame (common from side and diagonal angles, where both ends of the bar can be in view), the pipeline doesn’t just pick one. It combines the position signal from all the plates it’s tracking into a single, more stable estimate, rather than throwing away the extra signal. Exactly how that combination is weighted is still an active area of tuning, so this is intentionally the short version.

What Was Validated

The core validation dataset includes over 1,300 training videos recorded by real athletes, plus several hundred more from real app usage feeding the RQ2 research, nearly 1,700 videos in total. Squat, bench press, and deadlift. Multiple camera angles. Various loads from warmup to near-maximal. This is real-world data, not a controlled lab setup with a motion capture system, because the product needs to work in real-world conditions.

That corpus has been run through the pipeline a lot. Across every method-comparison sweep so far, the research has tested more than 5,000 distinct pipeline configurations (every combination of segmentation model, fitting algorithm, imputation method, smoothing technique, and a dozen other method choices, counted directly from the sweep definitions). Multiplied across each sweep’s video set, that adds up to easily hundreds of thousands of individual pipeline runs over the course of the project, not a handful of one-off tests.

The two headline claims rest on different evidence, and it is worth being precise about which is which, because they are not interchangeable.

Velocity and ROM agreement comes from the subset where a RepOne linear position transducer was attached to the bar at the same time as the video was filmed: 408 reps with a simultaneous, independent displacement reading to compare against, rep by rep. That is the cohort behind the ICC and MAE figures above, computed with the standard agreement metrics — intraclass correlation coefficient (ICC), mean absolute error (MAE), Pearson correlation, and Bland-Altman analysis.

Rep-count accuracy is checked across a much larger set, roughly 1,300 sets, and most of those are ordinary platform uploads where no tethered device was ever on the bar. There the reference is the athlete’s own reported rep count, not an instrument. That is a weaker ground truth than a transducer, and it is also a more realistic one: it measures whether the rep counter agrees with real people filming real sets in real gyms, which is the condition the product actually runs in.

Conflating the two would inflate both. The full breakdown, including per-exercise and per-angle results, will be published with the finished thesis.

What the Limitations Are

Honesty builds trust. Here are the real limitations:

Camera angle matters. The pipeline works best from side and diagonal views, where the weight plate is clearly visible and the bar moves across the frame. Front-facing angles produce poor results because the plate appears edge-on and the bar moves toward/away from the camera. This is a physics limitation, not a software one.

30 fps caps peak velocity resolution. Phone cameras typically record at 30 frames per second. A linear transducer samples at 500-1000+ Hz. The lower sampling rate means peak velocity (the highest instantaneous speed) may be slightly underestimated, because the true peak might occur between frames. Mean velocity (the average across the concentric phase) is less affected, because it averages over many frames.

Non-standard plates introduce calibration error. The perspective correction defaults to a 450 mm plate diameter (IPF standard for red 25 kg and blue 20 kg plates). The pipeline now supports custom plate dimensions if you’re using different plates. Without the correct diameter set, smaller plates produce a systematic bias in absolute values. Relative comparisons (rep-to-rep, session-to-session) are still valid regardless.

Squat is the hardest lift to measure. Among the big three, squat has the weakest accuracy, partly because squat bar speeds are higher (making the 30 fps limitation more impactful) and partly because squat setups often have rack uprights that partially occlude the plate.

These are real limitations, they’re documented in the thesis, and the pipeline is continuously being improved. Knowing them helps you use the tool well: film from the side, use competition plates when possible, and trust the trends more than any single data point.

Why This Matters for You

Most consumer VBT apps are black boxes. You get a number. You don’t know how it was calculated, what was tested, or what the error margins are. PuffinVBT is different: it grew out of academic research, is being validated against a known reference sensor on real training footage, and the full thesis (with methodology, complete results, and limitations) will be published when it’s finished. Until then, this post gets updated as the numbers do.

The numbers you see in the app aren’t guesses. They’re the output of a research-grade pipeline running on your gym clip, one that’s still being actively developed and improved. That doesn’t make them perfect (no measurement is, and the pipeline keeps changing as it gets better), but it makes them trustworthy: every number has been checked against a reference sensor, not assumed.

PuffinVBT includes 15 free analyses to start, no card required.

Sources

Recent, directly relevant work:

  1. Renner, A., Mitter, B., & Baca, A. (2024). Concurrent validity of novel smartphone-based apps monitoring barbell velocity in powerlifting exercises. PLOS ONE, 19(11), e0313919. Twenty competitive powerlifters, three smartphone apps, RepOne as one of the reference instruments, exactly this validation question asked by an independent lab.
  2. Pueo, B., Lopez, J.J., Mossi, J.M., Colomer, A., & Jimenez-Olmedo, J.M. (2021). Video-based system for automatic measurement of barbell velocity in back squat. Sensors, 21(3), 925.
  3. Nagao, H., & Yamashita, D. (2022). Validation of video analysis of marker-less barbell auto-tracking in weightlifting. PLOS ONE, 17(1), e0263224.
  4. Thomas, J., & Wahlström, J. (2025). Estimation of resistance training RPE using inertial sensors and electromyography. arXiv:2510.03197.

Statistical methodology (the standard references for the agreement metrics used above, current regardless of publication date):

  1. Shrout, P.E., & Fleiss, J.L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86(2), 420-428.
  2. Bland, J.M., & Altman, D.G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307-310.

Pipeline methodology:

  1. Gonzalez-Badillo, J.J., & Sanchez-Medina, L. (2010). Movement velocity as a measure of loading intensity in resistance training. International Journal of Sports Medicine, 31(5), 347-352.
  2. Halíř, R., & Flusser, J. (1998). Numerically stable direct least squares fitting of ellipses. Proceedings of WSCG, 125-132.
  3. Sturm, P., & Maybank, S. (1999). On plane-based camera calibration: a general algorithm, singularities, applications. Proceedings of IEEE CVPR.