Measuring What Matters

Plus: How I made an animated holiday short using AI, and why 2025 was the year AI changed design.

Hello friends, and happy new year! My apologies for getting back into the swing of things late, but life happens.

I’m trying a new format. A lot of times, when I’m reading a random assortment of articles across dozens of sources, I make connections. I connect the dots. This week on the blog, I linked to a bunch of articles about metrics—cold, hard numbers that quantify the outcomes we create with our design work. That’s this week’s big idea…

The Big Idea: Measuring What Matters

We’re drowning in data but starving for insight.

Product teams track everything now. Dashboards multiply. Every feature ships with its own set of KPIs. And yet, despite all this measurement, teams still struggle to answer basic questions: Is this working? Are we building the right thing? How do we know?

The problem isn’t a lack of data. It’s a lack of clarity.

Shipping faster doesn’t help if you don’t know why you’re shipping. AI tools promise to accelerate everything—PRDs, designs, code—but speed without direction is just expensive motion. The fastest teams aren’t the ones shipping the most. They’re the ones who understand what problem they’re solving and for whom. That understanding has to come first, before the metrics, before the dashboards, before the sprint planning.

Once you have that clarity, where do you look for signal? Not where most teams look. Surveys and interviews feel productive, but there’s a persistent gap between what users say and what they actually do. We all want to seem more competent than confused. Our memories are fuzzy about routine tasks. So when someone tells you what they want, they’re often solving for their own experience in ways that don’t generalize. The better approach: watch behavior, measure outcomes, and use what people say to add context—not to drive decisions.

But here’s where it gets uncomfortable. Even when we measure the right behaviors, some of the most important things resist quantification entirely. W. Edwards Deming put it plainly: the most important figures needed for management are unknown and unknowable. That doesn’t mean we ignore them. It means we stop pretending that ROI calculations capture the full value of good design. Sometimes the most powerful question you can ask leadership is the simplest one: How would you grade this product? When was the last time anyone just held it in their hands and looked?

When measurement does make sense, it should start with the user’s problem—not your feature list. The TARS framework offers a useful structure: What percentage of users have this problem? How many adopted your solution? How many stuck with it? How satisfied are they? It’s a way to connect what customers experience with what you’re tracking in the product.

Zoom out further and you need something to orient the whole product. A North Star Metric defines the relationship between the problems you’re solving and the value you’re creating. But picking the right one depends on knowing what game you’re playing. Are you competing for attention, transactions, or productivity? Different games demand different measures of success.

The throughline: data is a tool, not a destination. Clarity comes first. Watch what users do, not just what they say. Accept that some things can’t be captured in a dashboard. And when you do measure, make sure you’re measuring what actually matters—not just what’s easy to count.

---

Directing AI: How I Made an Animated Holiday Short

In my last email, I teased that I was working on something in ComfyUI:

I’ve been playing a lot with ComfyUI recently—I’m working on a personal project that I’ll share if/when I finish it. But it made me realize that prompting a visual to get it to match what I have in my mind’s eye is not easy.

It’s an animated short and I “premiered” it for my family on December 30th:

I’ve spent most of my career working with world-class human artists—animators at Pixar, art directors, motion graphics artists, and illustrators at ad agencies—so I was not entirely sure this could work or if I could pull it off. Over two months, I wrestled with tangles of ComfyUI noodles to direct a small army of AI video models to make a two-and-a-half‑minute holiday short about the thing I care about most: my family. I wrote up a story of how I did it—and what it really means to “direct” AI instead of letting it direct you.

---

The Year AI Changed Design

At the beginning of 2025, AI was still something designers were poking at from the edges—cute demos, promising tools, lots of hype. Twelve months later, it’s sitting in the middle of our work: in our research, our explorations, our prototypes, and in some cases, our production code. The fundamentals of design haven’t changed, but how we get from problem to product has—and 2025 was the year that shift became impossible to ignore. Here’s my wrap-up.

---

Something Fun

MTV Rewind

“I want my MTV!” That is the line that many music artists spoke to camera in a famous campaign by George Lois to get fans to call their cable companies to ask for MTV. It worked.

While MTV’s international music-only channels went off the air at the end of 2025, its US channels still exist. They’re just not all-music all the time like it was in the 1980s.

That’s where MTV Rewind comes in. It’s a virtual TV where you can relive MTV programming as it was. Built by an artist going by FlexasaurusRex, it’s an archive of Day 1 programming, and then different channels (YouTube playlists) to shuffle through the different shows, including 120 Minutes.

---

What I’m Consuming

Top 10 UX Articles of 2025. Good reference overall. This piece highlights the most-read NN/g UX articles of 2025, which focus heavily on how AI is reshaping UX practice, skills, and careers, alongside enduring usability fundamentals like button states, navigation patterns, and emerging visual styles. It also surfaces a bonus set of popular 2024 articles that deepen into AI for research, synthetic users, prompt design, and the limits and risks of generative chatbots. (NN/g)

The Enshittifinancial Crisis. Ed Zitron, in a 19,000-word opus argues that the AI boom is not a real, productive revolution but a massive, debt- and venture-fueled bubble that extends “enshittification” into finance itself, with banks, VCs, hyperscalers, and data-center builders all propping up unprofitable AI companies and opaque GPU spending to keep stock prices rising. He warns that once cheap capital and investor credulity run out, AI startups, data-center projects, and even parts of the broader market will implode, exposing how little genuine value and profitability lie beneath today’s AI hype. (Ed Zitron)

Click: How to Make What People Want. Jake Knapp, the designer who brought us the design sprint is back with a new book and new process. Click is a practical guide to starting big projects using a two-day “Foundation Sprint” that helps teams clarify their ideas, differentiate from competitors, and make smart decisions fast. It builds on the original Sprint framework with 10 lessons, a step‑by‑step playbook, and case studies from companies like Google, Nike, and Slack so you can quickly test a founding hypothesis and set your team up to win from day one. (Jake Knapp with John Zeratsky)