Truthset Comes Up With AAA Idea For Rating Data Quality
In the early days of computing, “garbage in, garbage out,” became a familiar phrase that explained that machines were only as smart as what people fed them.
Scott McKinley (Photo courtesy of Truthset)
A modern corollary as artificial intelligence is infused into all parts of the advertising industry is that “not all data is created equal.” It stands to reason that being the authority on data quality would put you in an enviable position.
Truthset, which has been in the business of evaluating data quality, is launching a new way to let people know what they're getting into when they dive into data.
In a deceptively simple way, Truthset is giving data providers grades like school children get from teachers, or bonds receive from rating agencies.
Oakland, California-based Truthset was founded in 2019 to validate the accuracy of consumer data.
Its new Data-Rated Audiences sorts the data offered by the more than 20 top data providers in the Truthset Data Collective by their precision, with AAA having the most fidelity, followed by AA and A all the way down to B, the lowest grade it gives.
“This is what the industry needs,” Truthset CEO Scott McKinley told TVRev. “There’s been no clarity, transparency, or knowledge of how good or bad all this data is. It's fueling everything here. And now we have those answers.”
Truthset used to score the accuracy of data on a 0 to 100 scale. If it looked at a data set of what were supposed to be Hispanic consumers, and it turned out there was a 72% chance each individual in the file was actually Hispanic, the data set got a score of 72.
“We found no one knew how to use that,” said McKinley. When you pull into a gas station, you might not know if your car will run best on 87 octane gas or if it needs 94 octane fuel. But you do know that your Toyota runs on regular and your Audi will break down unless you pump premium, even though that costs more.
“The new ratings are suggesting that different grades of data are appropriate for different use cases,” McKinley said.
Companies want high-end data for AI modeling or seed file creation. But less accurate data is OK for broad audience targeting.
You might be surprised by what qualifies for a passing grade in today’s data world. To earn a top AAA rating, data has to be above 90% accurate. But to be AA, data has to be around 65% accurate. A-rated data is 40% to 60% accurate and finally with B-rated data, the chances are a person in the set is not in your target.
Truthset dosn’t rate data that’s less than 30% accurate.
“The bottom 30%, it’s all a bunch of garbage. So I only want to include records from the data providers that are at least 30%,” McKinley said.
McKinley said that accuracy can be that low because of the pressure customers put on data providers to provide scale. To achieve scale, they loosen their algorithms, he said.
How does Truthset assess the accuracy of large data sets? It has its own patented algorithm that analyzes the providers’ data.
“We ingest about 5 billion emails every quarter. The emails have their information about people. We compare it against an independent truth set, which is a combination of panels and other deterministic data, and we arrive at these accuracy scores,” he said.
Why would big data providers submit to being rated? They not only submit their data, but they pay Truthset to be evaluated.
McKinley admitted Truthset was worried about what would happen when they told members of its collective how bad some of their data is. “It was painful,” he said.
But the fact is that top data customers are very sophisticated when it comes to data science, so they already have a good idea of which data is most accurate—and which are useless.
“The toothpaste was out of the tube. And the providers said ‘I'd rather be on the right side of this. I'd rather weaponize it for my own benefit. I'd rather understand how to leverage it,’” McKinley said.
Even within an individual data provider, it turns out some data sets may be great, but others not so good.
One data company found that when it tried to sell ethnicity data, the deals never closed. Working with Truthset let them know their ethnicity data was terrible, McKinley said. They had to decide whether to stop selling ethnicity data or make a material investment in ethnicity data if they wanted to be competitive.
On the positive side, if you get a good grade from Truthset, you can get a badge to post, put the score in your marketing materials and boast about being among the leaders in, say, household income data.
And data providers can charge more for their data that gets higher grades.
“At Stirista we’ve always prioritized data accuracy because it directly impacts performance,” Ajay Gupta, CEO and founder of Stirista, an identity-driven marketing data company, said in a statement.
“We’ve worked with Truthset to validate the strength of our demographic data because independent verification matters, especially in a crowded market. We were early to offer audiences with third party validation not because we needed it but because it reflects the high standards we already hold ourselves to,” Gupta said. “As the industry moves toward greater transparency we see this as essential to smarter, more effective marketing.”
The beauty part of Truthset’s business is that data buyers also pay to access the ratings.
“If you're a buyer, you now want choice,” McKinley said.
That choice isn’t limited to picking a single vendor with a high grade. If, for example, you’re looking to reach Black moms with Pampers ads, it’s possible to take just top-rated records from several different providers.
“It allows people to say ‘I'm gonna buy AAA, and I want it from all the data providers who are willing to sell me just their triple As,’” McKinley said. “So we basically help them by waving a magnet over 22 haystacks and sucking out only the needles.”
As data becomes a bigger part of the business, marketers complain about the growing cost of the so-called tech tax that means less of their ad dollars are actually going to buy media.
But McKinley argues that Truthset reduces the ad tax by ensuring that marketers use the data that gives them the best bang for their buck and reduces wasteful spending on impressions going to people outside of their targets.
Leading global brands have already piloted the solution, reporting substantial improvements in campaign performance and return on investment, according to Truthset.
All in all, the rating business can be a good one.
“You want to be the independent arbiter of truth for the quality of data that's powering a trillion dollar market. This, this could be a very, very valuable enterprise,” McKinley said.
“I was at Nielsen, so I learned how valuable it is if you're the company that rates things,” he said. “Nielsen had a market cap when it got taken private again at $16 billion for rating a $70 billion market. I mean, that's pretty incredible.”
McKinley also thinks that this is the right time for Truthset to be evaluating data.
“The whole ecosystem is like having an existential crisis around transparency and identity and accuracy, and it's just a big mess,” he said. “And now cookies are back,” he said, referring to Google’s decision to again delay getting rid of cookies as a way to identify people. “No one even liked them in the first place.”
Having an independent judge of data quality is good for the industry, McKinley said.
“Everybody can know what they're buying and selling. We expect pricing to go up for everybody, CPMs to go up for publishers, effective CPMs to come down for brands. It’s also better for the consumer, because they’ll get more relevant ads,” he said. “We're trying to snap everybody to a standard that's objectively accurate from an independent provider, to remove friction and obfuscation and bullshit from the data ecosystem.”
# # #
They call it digital dominance. A new report from the IAB forecasts that digital video will capture 58% of advertiser spending on TV. Last year digital video — including CTV, social video and online video — accounted for 51% of spending, topping linear TV for the first time. Digital video drew $64 billion in spending in 2024, up 18%, and is seen rising another 14% to $72 billion in 2025.
The IAB said that programmatic self-service tools are attracting ad dollars from small- and mid-sized businesses.
”CTV is making it clear it’s a go-to channel for both viewers and advertisers and is expected to continue growing along with social video and online video,” said IAB CEO David Cohen. “The video industry continues its transformative shift towards streaming driven by content, creators, technology, and improved measurement. However, it is important to acknowledge that ongoing economic uncertainty, including tariffs, geopolitical conflicts, and changing consumer confidence, the marketplace in 2025 is more difficult to predict than ever before.”
# # #
Speaking of CTV, a new report from Upwave and IRIS.TV finds that when it comes to brand building, contextually targeted CTV campaigns outperform industry benchmarks for branding. The report found that IRIS-enabled contextual campaigns generated a 2X lift in awareness, a 3X lift in ad recall and a 5X lift in favorability compared to norms registered by all Upwave campaigns.
The report highlights a campaign by Carl’s Jr. that outperformed 90% of the QSR campaigns on Upwave, showing big scores for recall, consideration, awareness, and favorability.
“What the data shows is compelling: aligning ads with content isn’t just intuitive — it’s highly effective. In today’s fragmented, competitive environment, IRIS.TV and Upwave are giving marketers the tools to maximize Brand Outcomes,” the report says.
# # #
Future Today, the company behind streaming channels including Fawesome, Filmrise and HappyKids, said it launched the Future Today Marketplace for CTV advertisers. The marketplace enables marketers to do addressable advertising in brand-safe premium inventory using contextual targeting from providers including IRIS.TV.
“Future Today Marketplace, elevates the value of our CTV inventory, across our popular services, by incorporating best-in-class technology, to deliver an end-to-end solution to our buyers that empower our advertisers with transparent, privacy compliant data and insights necessary to drive their KPIs in this rapidly evolving landscape,” said Tim Ware, VP of Future Today’s Marketplace.
Future Today said its marketplace is interoperable with ad tech platforms such as Magnite, PubMatic and Index Exchange and works with measurement producers including iSpot, NCSolutions and EDO. Campaign analysis can also be obtained through TVision and Sleeping Giant Labs.
“PMG clients have seen scale in reach and outcomes leverage first party contextual data and partners such as IRIS.TV," said Mike Treon, head of CTV and video strategy at PMG. "Our clients benefitted from the optionality to target and optimize to Future Today's significant scale across flagship services: Happy Kids, Filmrise and FawesomeTV."