--- title: "Your feature store is training your model on a lie | Snowplow Signals" description: "Feature stores key features on whatever ID you hand them. Five lies your model learns when that ID is only part of the person." source: https://signals.snowplow.io/blog/feature-store-training-your-model-on-a-lie --- [Blog](https://signals.snowplow.io/blog) / Industry # Your feature store is training your model on a lie AK [Agnes Kiss](https://signals.snowplow.io/blog/authors/agnes-kiss) , Data Engineer Oct 13, 2026 · 6 min read ![A puppet show. Above the stage, half in the dark, your customer holds the control bar. Three marionettes that look exactly like him act on the stage: one browses on a phone, tagged phone cookie, orders: 0; one thinks about a watch, tagged Safari cookie, first visit; one pays on a laptop, tagged laptop cookie, with an order count that keeps going up. In the front row, an audience member labeled “feature store” holds up notes reading “New customers: 1. phone guy, 2. new guy, 3. laptop guy.”](https://signals.snowplow.io/images/blog/feature-store-training-your-model-on-a-lie/puppets.svg) Feature stores have gotten very good at one job. You give them an ID, and a few milliseconds later they hand back fresh features for it. What they never ask is whether that ID is a whole person or one browser’s partial view of one. Answering it means [stitching identities](https://signals.snowplow.io/learning/customer-profiles-and-identity/what-is-identity-stitching) together at read time, without giving up the speed you bought the feature store for. That’s hard, and they leave it to you. Unless your site makes everyone log in first, the ID your feature store takes for granted is usually a cookie or a device ID. Your customers have a phone, a laptop, a work machine, and a browser that forgets them whenever its cookies expire or get cleared. A user ID only joins them up after they sign in, and leaves out everything they did before. So the feature store holds several half-profiles of the same person, and your model learns from them, scores them, and acts on them with complete confidence. ## Three problems feature stores ignore When the ID is only part of the person, three things go wrong. Each one results in training and serving a lie to your model. Let’s walk through five examples. ### 1\. Your model learns the wrong thing Training data built on half-profiles teaches the model relationships that aren’t true. Research looks like a lack of interest, loyal customers look new, and one bargain hunter looks like five. #### Lie 1 The signal flips: “People who research don’t buy” Sir Walter spends two weeks researching a $12,400 watch on his phone during his commute. He signed in on that phone once, months ago, but usually browses logged out. One evening he opens his laptop, signs in, and buys the watch in 90 seconds. ![Two panels of the same man with glasses. On the left, at night, he stands in a worn hoodie outside a watch store window showing a $12,400 watch, holding his phone, tagged “Walt,” phone cookie, sessions: 40, orders: 0. On the right, he stands on a yacht deck in a smoking jacket and bow tie, admiring the watch on his wrist, next to a laptop showing “Paid,” tagged “Sir Walter,” laptop cookie, sessions: 1, orders: 1. Caption: “Two rows in your training data. One millionaire.”](https://signals.snowplow.io/images/blog/feature-store-training-your-model-on-a-lie/beggar-billionaire.svg) Your training data gets two rows: a phone cookie with 40 sessions labeled “didn’t buy,” and a laptop cookie with one session labeled “bought.” Repeat that for every buyer who researches on one device and pays on another, and the model learns that heavy research predicts not buying. You can’t average this away, because the relationship itself is reversed. Lead scoring deprioritizes your most serious buyers, and nothing throws an error. #### Lie 2 The errors pile up on one segment: “He’s new here” Bernard has been a customer for five years and has placed 38 orders, but he only signs in at checkout, so his features are keyed on the cookie. He shops in Safari on his iPhone, which caps cookies set by JavaScript at seven days. Whenever Bernard skips a week, he shows up in your training data as a newcomer. The same happens to anyone who clears cookies, and those people aren’t a random sample of your customers. The model learns that iPhone users are new and low value, so it stops targeting some of your highest-value customers. Your validation set shares the bias, so it won’t catch it. #### Lie 3 The discounts get larger: “All our customers need a discount” Your site shows a 20% welcome offer when `is_new_visitor` is true. Maya, a regular customer, works this out. She opens an incognito window, then a second browser, then her phone, then her partner’s laptop. You’ve paid five discounts to someone who would have paid full price, and your offer-uplift model records five successful conversions. Next quarter it recommends offering larger and more frequent discounts. ### 2\. Your app acts on the wrong person At request time, your app fetches features with whatever ID it has. When that ID leads to a fragment, a nudge or a support agent acts on the fragment instead of the customer. #### Lie 4 The nudge goes to someone who already paid: “She abandoned her cart” Ann signed in on her phone once, weeks ago. Today she adds $140 of sneakers to her cart on it at lunch without logging in, then signs in and pays on her laptop that evening. The next morning, the abandoned-cart model looks up her phone’s cookie. ![Two panels of the same woman. On the left, labeled “Cart-abandoner Ann,” phone cookie, cart_abandoned: true, a phone screen shows her cart with $140 sneakers and a notification reading “You left something behind! 10% off.” On the right, labeled “Ann,” laptop cookie, order: paid, $140, she stands by her front door wearing the new sneakers next to the open delivery box, frowning at her buzzing phone, with a speech bubble reading “10% off?!” Caption: “Abandoned, says the phone. Delivered, says the courier.”](https://signals.snowplow.io/images/blog/feature-store-training-your-model-on-a-lie/cart.svg) Ann’s phone buzzes: “You left something behind! Here’s 10% off.” She paid full price yesterday, so she opens your website chat from her work computer, where she has signed in before but isn’t today, to ask for the 10% back. The support agent looks up that computer’s cookie, finds nothing about her order, and offers her a first-order discount. Her history exists, but the app asked for it with an ID that doesn’t lead there. ### 3\. The quick fix merges the wrong people Joining profiles that share an identifier looks like the obvious fix. Join them on the wrong one, and you create a customer who doesn’t exist. #### Lie 5 Two people become one: “Mom loves energy drinks” Mom and her teenage son both have accounts, and both use the family iPad, so you join their profiles on the iPad. Now they’re one person who wants knitting patterns and energy drinks, with a purchase history that belongs to neither of them. Your model learns from the combined history of two people, so what it learns matches no one who actually exists. ## Snowplow Signals with Identities: one person for training and serving Today you mostly get to pick between fast and stitched. Feature stores serve features in milliseconds, keyed on whatever ID you hand them. Customer data platforms stitch identities, either in real time as events arrive (Segment) or in scheduled batch runs (Hightouch, RudderStack). Some also expose profile APIs, with a trade-off. Batch-stitched profiles are minutes to hours old. Real-time profile APIs are rate-limited, and the counts they return over a rolling window, like orders in the last 30 days, are often computed in batch rather than updated as events arrive. A feature store doesn’t stitch identities itself, so when you build training datasets you might do it upstream, in the warehouse or with an identity resolution tool. That only fixes half the problem. You also need to stitch identities in real time when you compute and serve features. Without that, your model trains on whole profiles and offline evaluation looks great. But at request time your app still only has the cookie, or a partial profile, so in production the model scores fragments of the profiles it trained on. Online performance drops. That’s [training-serving skew](https://signals.snowplow.io/learning/feature-stores/what-is-training-serving-skew), caused by identity rather than code. Snowplow Identities stitches a person’s identifiers together. The Signals integration with Identities brings that stitching to your features, which Signals calls attributes. They’re computed on the whole person and read in real time with whatever identifier your app already has, so your model can train and serve on the same person. ![Diagram: the three puppets from the opening image, one swinging with a phone, one thinking about what to buy in Safari, one paying on a laptop, feed into Identities, which stitches their identifiers. Identities feeds Signals, which computes attributes on the whole person. Signals feeds both training and serving, and a bracket joins them to one face, labeled same person.](https://signals.snowplow.io/images/blog/feature-store-training-your-model-on-a-lie/one-person.svg) Next, when Identities later discovers that two IDs belong to one person, Signals will combine their profiles and compute the features correctly, including windowed features such as orders in the last seven days. ## How Signals solves for this | Problem | Lie | What fixes it | | ------------------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Your model learns the wrong thing | Lie 1: “People who research don’t buy” | One identity for training and serving | | Your model learns the wrong thing | Lie 2: “He’s new here” | Combining profiles exactly when IDs merge | | Your model learns the wrong thing | Lie 3: “All our customers need a discount” | Collecting an identifier, such as a login or an email | | Your app acts on the wrong person | Lie 4: “She abandoned her cart” | Reading a profile by any linked ID | | The quick fix merges the wrong people | Lie 5: “Mom loves energy drinks” | Never merging IDs that belong to different people, with user\_id marked as a [unique identifier](https://docs.snowplow.io/docs/identities/concepts/unique-identifiers/) in Identities | ## Try it on your own events The Identities integration is coming soon. Until then, [start free](https://signals.snowplow.io/start) and see what Signals builds from your own events. The free tier runs on the real engine, needs no card, and shows an attribute updating against your own traffic in the first session. AK [Agnes Kiss](https://signals.snowplow.io/blog/authors/agnes-kiss) Data Engineer, Snowplow Follow Agnes for more on real-time customer context. [LinkedIn](https://www.linkedin.com/in/agneskiss123/) [More from Agnes → ](https://signals.snowplow.io/blog/authors/agnes-kiss) Start on the free tier Add the SDK and watch an attribute updating against your own traffic in the first session. The free tier runs on the real engine: no card, no sales call. [Start free](https://signals.snowplow.io/start) Share this post ✕ Click the image to zoom in. Esc closes. Scroll to pan. Click the image to zoom out. In this post * [Three problems feature stores ignore](#three-problems-feature-stores-ignore) * [Snowplow Signals with Identities: one person for training and serving](#snowplow-signals-with-identities-one-person-for-training-and-serving) * [How Signals solves for this](#how-signals-solves-for-this) * [Try it on your own events](#try-it-on-your-own-events) Start on the free tier Add the SDK and watch an attribute updating against your own traffic in the first session. The free tier runs on the real engine: no card, no sales call. [Start free](https://signals.snowplow.io/start)