As someone who believes that ad tech is the primary force driving the ever growing consumer distrust of technology, I can’t help but feel a little giddy reading this story.

I&#x27;m certain they do want to find out.  Adtech still wants to know where paying users are even if the total is less than they&#x27;d like to claim.

&gt; overwhelming majority of digital ad formats just don’t workDoesn&#x27;t it? It may have miserable conversion rate, limited target audience but it apparently is commercially viable. People would hardly spend money on sending e-mail spam in 2020 if it didn&#x27;t work.

Facebook and IG ads work. I agree that display network ads do not.

Mobile devices tend not to have adblockers, so exposure might be a lot higher.Also, the only ads I&#x27;ve clicked are mobile ads. I&#x27;ve even installed apps via ad before - I&#x27;ve never once purchased an item from an ad online (but I&#x27;ve also run an adblocker for a decade).Ads certainly &quot;work&quot; by some metric, and some definition of &quot;work&quot;, but I think the metric and definition is unclear.

Instagram ads for millenials are like flies to honey in my anecdotal experience. So many of my peers buy their furniture, dress shoes, clothing, water bottles, watches, dog leashes, pretty much everything in their lives from seeing some like-aged person use that thing on an instagram ad (and the product usually being pretty cheap). It blows my mind how successful these dime a dozen drop shipping companies based on instagram are with millenials and gen z, and buying like this will only be more normalized in the future as more people know someone who got something nice from that instagram ad.

Digital advertising is super effective for mobile ftp games. It is not a question if they work, it’s a question if the game can earn enough to justify the cost. Companies in that space have a super smart team of economists and data scientist constantly checking when a mobile app is worth advertising.Someone in this thread said no one has proved to them that digital ads are ever worth it. My experience is that when they are you don’t want to advertise it to potential competitors or the people who carry your ads.

I think the problem is that the overwhelming majority of digital
ad formats just don’t work. Does anyone who works in digital ad tech genuinely believe that these formats work? I have been here for the last ten years and have come to believe they do not, certainly not comparable like traditional linear TV or print in their heydays.There are exceptions. I think Google search ad works so well it’s eating the product. I think YouTube’s TrueView is great for modern brand advertisers (safety issues aside). But by and large, banner ads, forced video pre-rolls, etc don’t work. The negative message of the format offsets any message you try to convey.Yes, fraud is a part of this. But I think a lot of this evidence looks the same as if none of their targeted, performance-based advertising was doing anything at all, beside really weak branding.Genuinely interested to hear from people who disagree.

Interestingly mobile ads are typically cheaper to buy than desktop ones.

I&#x27;ve been playing a silly phone game for about a month now. Only last week I realized it has ads: I&#x27;d become so ad-blind that I completely ignored the banner ad they keep at the bottom of the window at all times. I only noticed it because it flickered when transitioning to another ad.

Makes sense to me.CEO: &quot;We need to do something about growth!&quot;Marketer: &quot;Mobile ads are something!&quot;CEO: &quot;I read those are hot. Here&#x27;s a bucket of money! Do lots of something so we have lots of growth!&quot;[spending ensues]As the Uber story shows, a lot of money gets spent without anybody making sure that it actually works. The actual goal is not to have an effect. It&#x27;s to be seen as taking bold action. If the business gets better, bonuses and promotions ensue, no matter what the actual cause. Cargo cult management.It sounds insane, but I&#x27;m sure it&#x27;s rife. Years ago I was coaching at a Lean Startup weekend class. One team really killed it, rapidly testing product hypotheses through real-world testing. Their initial idea was quickly proven worthless, but they listened to users and came up with an idea that would sell.I kept in touch with one of those team members. He went back to his prominent, well-funded startup, excited to improve his company. But nobody wanted improvement. Execs wanted to sit in a big room, think big thoughts, and produce specs. The engineers were to implement those specs. But nobody would measure the effects of anything. Proof that a grand poobah&#x27;s idea didn&#x27;t actually pan out was deeply unwelcome. It slowly drove my pal mad and he quit. The company bumped along for a few years to a modest acquisition, but it never really lived up to the hopes.

Even if you do this the ad-people will claim that showing people ads repeatedly will make them buy your product eventually. Maybe when they actually need such a product.
At this point you cannot connect the events anymore.
This effect is probably weak but not falsifiable.

Engaging random marketing consultants is a good way to waste money on ads.However, anyone engineering for advertising from the start of your distribution plans should be able to measure these things. It’s not hard to track which signups came from ad clicks versus organic traffic. The problem is that most companies start out focused on their apps and simply assume advertising is a totally independent function that happens elsewhere in the company. Invest some time into including advertising as part of your signup process and it’s really not that difficult to track LTV of customers who signed up via ads.In fact, that’s how Uber confirmed this fraud. They noticed that some of these fake customers were clicking on the ads and getting signed up within seconds, which is something only bots can pull off. If the customers you’re getting from ad signups aren’t actually spending money in the app, you should know about this from basic analytics.

I have this suspicion that mobile ads have way weaker effect than claimed. No-one in adtech has any interest in finding out. Marketers don&#x27;t want to lose budgets or importance either. So a lot of money is wasted on annoying people. How sad.

I just don&#x27;t want to lose the web along with the ad tech industry though.

Impending crash - time to short the entire ad tech industry.

The ad industry has always claimed that fraud doesn&#x27;t matter because it is priced in. It&#x27;s been over a decade since I worked in click fraud detection but I believe Google&#x27;s numbers were around &lt; 2-5% of clicks were fraud (someone here might have up to date numbers). So this is a huge revelation from my perspective.

&gt; Also worked in ad tech. There were some types of customers that cared about performance, but the largest were typically agencies and all they cared about was spending the budget they were given.Large companies call it &quot;brand&quot; spending. Which is to say they&#x27;re spending to maintain brand awareness. Measuring ROI would call the practice into question.

&gt; From the tweet, it’s crazy that they were spending 100m without tracking the quality of the users.They were high quality users: Uber didn&#x27;t pay for installs, but for installs + first ride. What&#x27;s being alleged is that the ad networks were detecting a user installing the app, and fraudulently creating an ad display event to make it look like the install happened due to the ad.

Working in iPhone games, our biz dev people made us integrate with the shadiest of ad publishers. But we required attribution and could track how much the users spent, so could stop spending on them quickly, which was the normal result. Most of the companies were surprised we wanted attribution and had to build something custom, which to me means at the time nobody else was tracking it so lots of customers must have been getting fleeced. From the tweet, it’s crazy that they were spending 100m without tracking the quality of the users.Also worked in ad tech. There were some types of customers that cared about performance, but the largest were typically agencies and all they cared about was spending the budget they were given. Everybody knows this is happening, you can differentiate yourself slightly by doing better detecting it. But at the end of the day people really care about customer acquisition cost which includes the fraud, so if you had 0% fraud everyone would just raise their prices or margins

Depending on the content of the ads, I could easily believe that anyone who is remotely the market for even the occasional Uber ride already has the app on their phone and, if they want a ride will either use Uber, whichever ride-share is cheapest, Lyft if it&#x27;s available because they don&#x27;t like Uber, or a taxi&#x2F;limo in situations where that&#x27;s more convenient.I guess you can offer promotions but then you&#x27;re giving money away.

Wow, that’s quite a revelation. Uber turns off 2&#x2F;3 of ads and metrics are the same. Eventually, with optimizations, they got down to 10% of previous spend without loss of revenue. I wonder if the Uber story is unique or it cuts across all advertisers. How would this affect Ad Tech industry?

That theory can easily be tested by turning off the ads in question and seeing what actually happens. Can always turn them on again later if needed.

The solution to this would be to have a very authoritative site for a query show above any ads.

DHH of Basecamp had a long twitter thread about this. Their competitors bought “basecamp” as a keyword for google ads, effectively forcing basecamp to pay to stay #1 on searches for basecamp.Google has&#x2F;had no policies restricting this, unless of course, you guessed it, the keyword you are squatting is “google”.

This being allowed shows the sad state Google is in - you search for a brand, get ads for a competitor? Search engine, my ass. You now effectively need to pay Google to keep your product relevant through ads, even when the user is specifically looking for you.

In fairness, I&#x27;ve worked at several places that engaged in this practice, and they&#x27;re perfectly clear that they&#x27;re paying for clicks that they&#x27;d likely get for free.The theory is that if they don&#x27;t do this, their competitors buy this ad space and steal some of their organic traffic, and&#x2F;or that they&#x27;re just bidding up the cost of that space to make their competitors less efficient.Now, that theory may not be cost-efficient in practice. But it&#x27;s not like the marketers didn&#x27;t understand that someone searching for the name of our app were going to see it in the list anyway.

With google specifically, I&#x27;ve always wondered if the placement of the listing itself is related to the ad spend.

Isn&#x27;t it &quot;take away the ad&quot; and Google will put first a ad link to a competitor instead?

I&#x27;m not surprised by this a all. I do PPC for a living and I find that a lot of ad spend is totally wasted on paying for clicks that you would have gotten anyway, especially on Google and on Mobile devices.A lot of big brands bid on, and pay for, their brand terms because someone at some point told then that they should. So when a person uses search to find Uber, they get an ad first, then a regular listing second. And almost everyone clicks the first link. Take away the ad and the free click is first.

Yeah so that’s the weird part that I didn’t mention. These ads only run in the US.

I also had a small Google ad spend ($350 over 3 weeks), that I setup to run exclusively in the USA, and had the same experience where IP addresses recorded by hotjar were all from India, all new signups from the ad spend were &#x27;.gmail&#x27; email addresses and although I had about 100 signups not a single one entered a credit card.

With Google ads you can select the countries where you want the ads to run in. If you go worldwide and chose to maximize clicks, Google Ads will mainly run them in India and other emerging countries because it&#x27;s much cheaper to get clicks from there.

I’m not sure if this is equivalent to fraud. But I’ve recently discovered that 99.9% of the website visitors to my new startup’s website are just random @gmail addresses from India and China - most of which are coming from Google ads. We don’t spend much, about $100&#x2F;m mostly to keep our #1 ranking in Google (which I think helps), but it does seem fraudulent. If I turn the ads off, the traffic from those IPs drops off a cliff.

On the other hand, if you pay someone $20 to buy a brand new ApplePhone 12 XS Max, you should expect to not get the phone. The Twitter didn&#x27;t mention anything about price; and it is possible that the fraudsters were actually charging a normal market rate; and nobody deserves to be defrauded for trying to get something for cheap; but if Uber was taking the cheapest option then it&#x27;s hard to feel sympathetic towards them - they&#x27;re a big company with lots of marketing people and they should know better.

&quot;You get what you pay for&quot; doesn&#x27;t mean you deserve to be defrauded for choosing the cheap option.

In reality, you get AT MOST what you pay for. There are plenty of cases where you get much less, or nothing at all, for what you pay for.This story goes to show that.

I think Uber is to blame here, reading the story it is obvious that they bought ads from very shady sources&#x2F;networks, probably they went for the cheapest CPC. You get what you pay for.

That&#x27;s what happens when your own job is to waste oxygen, pull numbers out of thin air and take credit for &quot;conversions&quot; that were organic in reality; in this case you have no incentive to optimize things because you&#x27;d inevitably show the company your own position and maybe entire department is irrelevant and a waste of money.

No, it&#x27;s what happens when the objective is something different. The objective is often to get big ad views &amp; audiences, not to drive performance.

That&#x27;s what happen when you have too much money to spend and you don&#x27;t take time to evaluate if money is well spent.

I really can&#x27;t feel bad for someone who wants to pay 100M to annoy people and waste their time. The &quot;fraudsters&quot; here have done a service to society: Uber wasn&#x27;t satisfied with the free, organic growth they were getting and wanted to be more greedy and pay 100M to litter the web with ads; the fraudsters took the 100M and cleaned up their littering by &quot;consuming&quot; these ads with fraudulent clicks while giving Uber the outcome they wanted, albeit one they would&#x27;ve gotten anyway had they not been greedy.

I was VP Growth for a fintech for a while. And the one thing I learned that Paid Ads are a Trap. Here my open the read medium article about this <a href="https:&#x2F;&#x2F;medium.com&#x2F;@franz.enzenhofer&#x2F;ads-are-a-trap-80df01d2fbaf" rel="nofollow">https:&#x2F;&#x2F;medium.com&#x2F;@franz.enzenhofer&#x2F;ads-are-a-trap-80df01d2...</a>Tl;dr: money is at best a halfway decent accelerator. And as long as you don&#x27;t massive positive retention just a sinkhole of fake growth. And if you have this massive retention there are better ways to invest your money.

Exactly. That’s what the episode says too. The only people who could run the experiments hinge their entire career and industry on the results.

&quot;It is difficult to get a man to understand something, when his salary depends upon his not understanding it!&quot;

&gt; Why wouldn&#x27;t someone jump at an opportunity to prove the thing&#x2F;service they provide actually works, unless they were unsure about it themselves?Because everyone is already acting like they know it works, so the only way that experiment can change things is in a way that&#x27;s bad for the person in question. In that situation, they should (from a local, selfish perspective) be resisting even if they&#x27;re awfully sure it does work (and perhaps even if they&#x27;re right!).Given that, I don&#x27;t think the behavior has already given us an answer.

I work in the field. Incrementality testing is a big part of marketing measurement at any reputable agency. Any claims to the contrary are FUD.That said, companies like P&amp;G, Airbnb, and Uber, which are oft-cited as examples of digital not being worth it, fail to understand their own brand recognition and organic power, built through prior marketing efforts, as key to their current standing.Sure, TODAY, it doesn’t have the impact they’d like it to have but the investments PRIOR were key to ensuring their success.

I used to work in this industry.A small tech team investigated fraud on our platform and developed a system that was pretty robust at detecting and potentially shutting it down. But literally nobody was interested - even the people advertising don&#x27;t want to know.The people spending money are typically networks, media buyers, ad agencies, etc, far removed from the actual brand.There are so many parties who want a slice of the brand&#x27;s cash that they are all long past caring about whether the ad is being viewed by a human or not.

The experments are run.  Why don&#x27;t you look at the work that comes out of the Advertizing research foundation.

Uber was just incredibly incompetent to not audit their ad spend at all.I worked at an ad company. It was an absolutely standard metric to eg geo-fence ads out of a state or two for 3 months to demonstrate the impact of ads. This isn&#x27;t easily externally visible, but tests like this are standard practice.Particularly in the app install space, which is sketchy as hell once you stop buying from the top handful of vendors, buyers should be auditing by a couple million in annual spend. To get to $150m without looking hard at big chunks of their spend is just plain arrogance and&#x2F;or incompetence.

It’s like saying that no one who works on reliability of the systems is willing to run experiment to throw 100% errors for a month, because running experiment like that may show that reliability of the systems doesn’t matter.And then using data from a one system that went down and nothing happened as a proof that systems reliability doesn’t matter at all, and it’s huge scam by engineers.

&gt; Why wouldn&#x27;t someone jump at an opportunity to prove the thing&#x2F;service they provide actually works, unless they were unsure about it themselves?Because advertising in some form certainly works. If you can determine that approach &quot;A&quot; that everybody is doing is actually a waste of money but approach &quot;B&quot; is effective, then you can develop services around approach &quot;B&quot; and market them based on these findings.

&gt; The people that would have the power to run this experiment have their entire careers depending on things staying as-is.Any large company could invest in some experimentation, whether their marketing directors want it or not. It makes sense that at least a few do but just don’t publish the results

This is no industry with as much snake oil BS as adtech.It also happens to be the sole form of revenue all of the largest tech firms enjoy.No wonder there hasn&#x27;t been a real audit.Also, all those marketing execs buying all the ads would be out of a job too, so they aren&#x27;t speak up either.

Yep. Like the old adage goes, You cant make a man understand something, if his job depends on him not understanding it.

&gt; The unwillingness of anyone to run such an experiment is already an answer. Why wouldn&#x27;t someone jump at an opportunity to prove the thing&#x2F;service they provide actually works, unless they were unsure about it themselves?Or they have run the experiment and the results haven&#x27;t lines up with their own personal biases so they were discarded.

&gt; No one is willing to run the experiments necessaryThe people that would have the power to run this experiment have their entire careers depending on things staying as-is. Running the experiment carries a significant risk of exposing that the advertising operations they&#x27;re responsible for provide much less ROI than they pretend it does.The unwillingness of anyone to run such an experiment is already an answer. Why wouldn&#x27;t someone jump at an opportunity to prove the thing&#x2F;service they provide actually works, unless they were unsure about it themselves?

“Half the money I spend on advertising is wasted; the trouble is I don&#x27;t know which half.” - John WanamakerHalf was true in his time (a century ago) but I think our numbers are way worse.

&gt; No one is willing to run the experiments necessary.Not without reason. Even without the conflict of interest that Nextgrid points out in a sibling post, there&#x27;s still a significant financial barrier to attempting to measure this stuff. According to a former professor of mine who spent a large chunk of his career studying this stuff, the size of study you need to conduct in order to get any kind of statistical power at all on an ROI study is just absurd. See, for example, the treatment starting on page 15 of: <a href="http:&#x2F;&#x2F;www.davidreiley.com&#x2F;papers&#x2F;OnlineAdsOfflineSales.pdf" rel="nofollow">http:&#x2F;&#x2F;www.davidreiley.com&#x2F;papers&#x2F;OnlineAdsOfflineSales.pdf</a>

Alternatively, they dumped hundreds of millions of dollars into ads based on a wildly unrealistic notion of a customer lifetime (much like the rest of Silicon Valley).

This made the rounds in adtech in 2016 and it&#x27;s really nothing special. His entire speech comes down to a single quote: &quot;media will be quantified&quot;.Sure, everyone wants that. Precise attribution has been the holy grail for a long time and the struggles are far larger than just a few technology products. The new battleground is first-party data and clean rooms vs privacy regulations. And that&#x27;s after dealing with all the politics and perverse incentives that happen in such a massive industry.

Gabriel Leydon of Machine Zone spoke to the general topic at Code&#x2F;Media back in 2016. Basically discussing that they&#x27;d gone through the trouble of building internal expertise and tools for optimizing their ad spend to better ensure specific outcomes they desired&#x2F;required in a way that would inevitably lead to more sophisticated ad buyers and putting a nail in the coffin of traditional media advertising.<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=oXBqzpExvrk" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=oXBqzpExvrk</a>

&gt; No one is willing to run the experiments necessary.Or, no one is willing to share the results of the experiments.

Come up with some new product that requires some personal data for usage (eg. age, gender&#x2F;sex, address). Start to advertise this in just one country to one demographics, and look how many out-of-target orders you get.Maybe it&#x27;s even enough if you simply just sell it via mail order, you can then look at the addresses.There&#x27;s probably a natural information spread in any market (word of mouth, trade magazines), and there&#x27;s probably a physical dispersion of the target group of people too (people move, visitors&#x2F;tourists saw the ad&#x2F;product and order it at home), but it still should be a valuable to see how much effect just one campaign has.Maybe one of the best products for this could be a car. They are pretty standard, really not much difference between them, they are in all price ranges, and regularly new models come out. Advertise one in a few major US cities but don&#x27;t in others.

Traditional brand advertising testing (TV, newspaper, etc.) would be geography segmented as I understand it. So half the cities got the campaign and half didn&#x27;t. You can mimic that with IP based geo-location although you&#x27;d get more leakage than pre-internet.

It&#x27;s unclear how this experiment would be done. In the case of brand advertising, it&#x27;s likely that brand awareness would decay over some period of time and in turn purchase behavior would change.It&#x27;s not currently possible to run an A&#x2F;B experiment with a hold out group of potential customers across all channels, let alone for any longer duration experiment. So how can we separate cause and effect? (although pay per conversion channels do get gamed left and right)

Owning the pit into which other people throw their money seems like it would be pretty lucrative.

Why does reddit have ads if all ads are a money pit?I understand you no longer work there, but ads started in 2009 I believe, so you perhaps had some input on this?

But then why do they keep doing laughably bad advertising?

This is incorrect. Companies do the experiments. They just don’t publish the results. Why would they? The idea that Amazon doesn’t know the ROI on their ad&#x2F;marketing spend is laughable.

But someone did. &quot;Brand awareness&quot; is a thing.For example we can probably agree that for completely new companies spending on ads makes sense. Or giving out free samples, etc.Similarly for big companies doing media campaigns to keep the new ones at bay makes some sense.Even if word of mouth is a thing, even if there are organic searches, and even if it seems like a race to the bottom if everyone just tries to outspend each other.It&#x27;d be great to make experiments about how to sensible prevent&#x2F;regulate this ad arms-race. But first better data privacy laws.

Oh look, yet another news about how the advertisement economy fucks up more shit than it benefits:<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=25624112" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=25624112</a>“URL shorteners set ad tracking cookies”

I’ve thought and said so before: The ads racket is be a case of the emperor with no clothes, at best.At worst, it’s a front for building “profiles” of everyone.I can’t recall the last time I bought anything -because- of an ad.If anything, ads have sometimes actually put me OFF from buying something!

For brands that large the majority of their spend isn&#x27;t campaigns specific to new products, but overall brand management. The experiments would need to run for years or decades and if the prevailing belief that these are Red Queen&#x27;s Races is correct, the cost if they&#x27;re wrong could be the whole company.

Fortune 500 is a cesspool of insane inefficiency balanced by equally ~~insane rent seeking~~ insanely secure revenue. The sooner everyone understands this, the better.

The trick isn&#x27;t figuring out whether advertising in general influences people&#x27;s behavior. The trick is in figuring out if any particular advertising campaign generated more profit than it cost to run.Also, there&#x27;s one alternative that&#x27;s often forgotten in these discussions: Perhaps game theory is at play. It may be, for example, that, across entire industries, advertising costs more money than it&#x27;s worth. But that everyone has to do it anyway, because anyone who chooses not to will start losing ground to everyone else. IOW, just like in the standard prisoner&#x27;s dilemma, choosing to act is less about increasing your potential gains than it is about limiting your potential losses.There is an interesting long-running natural experiment in the pharmaceutical industry that suggests, albeit inconclusively, that this is the case.

Part of the claim is that the people who are checking are ad execs who, if PepsiCo stopped buying ads, would shortly be out of a job (or have their budget and influence slashed). A counter to this might be that different advertising channels are likely not identically effective, and a TV ad exec has a big incentive to poke holes in non-TV ads.

How do you look into it though? I don&#x27;t know, so i&#x27;m asking - but my immediate thought is that you treat it like science. You isolate an environment, advertise, and see if it has an affect. But the implication there is that if it doesn&#x27;t have an affect, money is on the table.If this is even remotely close to reality then it makes sense to me. Companies are more concerned with constant growth than strict efficiency, imo. They&#x27;re throw as much money around as possible, and every cent lost or left on the table is panic inducing.I also imagine different types or products and&#x2F;or markets behave quite differently. Eg a new product might very well benefit from advertising - since no one can buy your product or visit your store if they don&#x27;t know it exists.

&gt; Google had a clean-room data center specifically for pepsi&#x2F;frito.Why?

&gt;&gt; The idea that no one actually checked if this system works is simply not possible.Have definitely heard that one before. No disrespect.

I don&#x27;t understand this line of reasoning. P&amp;G, Unilever, Cocacola, etc have never, not once in history, had a gung-ho C level exec who said &quot;Screw it, I&#x27;m going to find out if our advertising works&quot;. And then either found it works and kept spending, or found out it doesn&#x27;t work and saved literally billions of dollars.There is so much money at stake that could be either saved or generated, its simply not possible that no one has looked at it. I used to help Pepsi&#x2F;Fritolay set up tracking to tie advertising on youtube to in-store sales. They spent millions of dollars to measure their ads, Google had a clean-room data center specifically for pepsi&#x2F;frito. The idea that no one actually checked if this system works is simply not possible.

Yes, the advertising industry is just obfuscated basic income for the twitter class.

&gt; No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary.Depends enormously what sort of business you&#x27;re in. I used to work for a company where all of our sales came from ads, 100%. It was trivially true that if we stopped advertising we would have no sales. We were also committed to running experiments: we knew how well all of our many advertising channels performed, and we ran A&#x2F;B tests for every change.

Listening to the podcast just a minute ago as I found this post... What a coincidence. I second that recommendation.

Great subject matter, but boy does reading the transcription remind me why I hate podcasts. Stop infantilizing the audience with these coo-coo-ing sounds bites and get to the damn point already!

These experiments are run all the time even before the internet. I remember reading about how for broad brand advertising they used to segment by city. Half the cities got an ad for Coca Cola and half didn&#x27;t. Then you compare sales of Coca Cola. No one will publicly publish numbers because it&#x27;s a mix of sensitive sales data and competitive advantage (ie: otherwise your competitors don&#x27;t have to burn money running their own tests). Also, smaller shops turn their online advertising campaigns on and off all the time to test impact.

Read the podcast transcript. An economist proposed turning off print ads in one market to find out if they mattered, and the head of marketing said he’d rather not know.

&gt; No one can actually prove it has any ROI at all.That&#x27;s a philosophical question of whether you consider statistics to be &quot;proof&quot;.&gt; No one is willing to run the experiments necessary.You&#x27;re nuts if you think this is true. I assure you that companies in traditional industries (i.e., without venture capital) can and do run these experiments.The Uber story is about venture capital and its anti-market incentives, not about the ad industry.

Tesla&#x27;s marketing spend is whatever it costs them in legal fees and fines to keep the mouthy celebrity CEO - and their high-profile campaigns in 2018 seemed pretty effective.

Telsa is in an enviable position of selling most of their cars before they&#x27;re produced. In that position, you don&#x27;t really need to advertise much. They also get a lot of PR to keep up brand awareness.If Hyundai couldn&#x27;t keep Genesis vehicles on the dealer lots, they&#x27;d advertise them less too. Having a dealer network means dealers that want manufacturer support in advertising to keep dealers happy, even if the new cars sell themselves, dealers need to get people in to sell used cars.

&quot;No one is willing to run the experiments necessary.&quot;Tesla is one natural experiment about not spending money on advertising in the mass media compared to traditional car companies that spend HUGE amount of money advertising.<a href="https:&#x2F;&#x2F;www.motorbiscuit.com&#x2F;gm-spends-an-embarrassing-amount-of-money-on-advertising&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.motorbiscuit.com&#x2F;gm-spends-an-embarrassing-amoun...</a>&quot;Hyundai spent $4,006 per Genesis vehicle sold in 2018. Ford’s Lincoln brand came in second with $2,106 per vehicle sold. After Jaguar and Alfa Romeo, GM’s Cadillac brand came in fifth with $1,242 spent per vehicle sold. Tesla was the lowest at just $3 spent per vehicle sold.&quot;

&gt; there was no change in buying behavior.I suspect that if our browser isn&#x27;t blocking ads then our brain is. It&#x27;s complete conjecture, but I assume that adblockers eliminate this cognitive load explaining a portion of why they became successful before they were necessary for security&#x2F;malvertising.

Beware extremes like &quot;no one can actually prove&quot;. One difference between internet ads and their more ethereal tv and radio predecessors is that adviews, clicks and purchases can be tracked. Also, there are techniques that can make even TV, radio, print and even digital to physical world ad performance more visible: coupons, response codes and campaign-specific phone numbers and URLS. That Uber was buying hundreds of millions in ads and could not attribute performance (sales, for example) speaks more to poorly designed campaigns and potentially very bad actors in the supply chain.

This story was on the Freakonomics podcast too. It definitely seems like a bubble, but there&#x27;s no external pressure to make the bubble pop. With the mortgage crisis, eventually, people can&#x27;t make the mortgage payments and the bubble bursts. But with people happily spending money on ads that don&#x27;t work, there&#x27;s no external pressure to stop. Will this bubble ever burst?

There was a good piece of reportage on this subject last year:<a href="https:&#x2F;&#x2F;thecorrespondent.com&#x2F;100&#x2F;the-new-dot-com-bubble-is-here-its-called-online-advertising&#x2F;13228924500-22d5fd24" rel="nofollow">https:&#x2F;&#x2F;thecorrespondent.com&#x2F;100&#x2F;the-new-dot-com-bubble-is-h...</a>

Back in 2017, Uber sued Dentsu for ad fraud<a href="https:&#x2F;&#x2F;www.japantimes.co.jp&#x2F;news&#x2F;2017&#x2F;09&#x2F;19&#x2F;business&#x2F;corporate-business&#x2F;uber-sues-dentsus-fetch-40-million-alleged-click-fraud-right-wing-news-site-breitbart&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.japantimes.co.jp&#x2F;news&#x2F;2017&#x2F;09&#x2F;19&#x2F;business&#x2F;corpor...</a>

There&#x27;s also the bit about luxury (mostly) car commercials. Half their purpose is to remind you that you made a good decision buying your &lt;brand&gt; car, and maybe your next car should be the same brand.

I&#x27;m only about halfway through the first of the podcast episodes you linked (thanks!), but just thinking about this logically for a moment:I would not be shocked to learn that advertising for a specific product or event is not particularly effective. However, I&#x27;m inclined to believe it has a huge effect on overall brand recognition.Let&#x27;s say you go to Amazon to buy a roll of toilet paper. How do you choose from the literally hundreds of options? You could spend a day of your life reading reviews, and trying to parse which ones are fake. Or you could buy the toilet paper from Scott because you recognize the brand.As I see it, buying brand advertising is a lot like buying an expensive suit. It&#x27;s not that the suit makes you more productive, but it is a sign of professionalism, and—frankly—of wealth. If a brand is advertising everywhere, you know they aren&#x27;t a fly-by-night company, and their products likely meet some standards of quality.

Additional resource discussed in the past:<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=21465873" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=21465873</a>

I&#x27;m a fan of the podcast but one argument they cited seemed to have a pretty glaring error - they looked at the case where eBay was comparing incremental gain on search ads over no ads. It&#x27;s methodologically hairy because eBay is a very major player with significant brand recognition.&quot;When Tadelis was working for eBay, the company was in the practice of buying brand-keyword ads. Which meant that if you did an online search for “eBay,” the top result — before all the organic-search results — was a paid ad for eBay.&quot;This doesn&#x27;t show that advertising doesn&#x27;t work per se, it shows that eBay didn&#x27;t hire a competent ad buyer. Whether or not you can prove the efficacy of advertising as a whole, this is not a valid approach.

Yes of course they can. They can run trials in different areas, and see if there’s an increase in spend. They run tests all the time.

it’s funny how trillions of dollars in market cap are built on top of some service with unprovable and questionable value.

But the experiments you run pit one ad type or channel against another. Have you ever run any experiments comparing ads to no ads?

<pre><code> PPC you only pay if the user actually engaged with the ad
</code></pre>
Who defines that &quot;engagement&quot; legitimately occurred? The ad network.Example: TVguide.com&#x2F;listings. On most devices, I get a nondismissable audio ad when I&#x27;m looking at telecast schedules. I kill the audio immediately.I never experience the bulk of the ad, and even if I can identify the product being advertised, I&#x27;ll form a strongly negative impression of them.I don&#x27;t count this as &quot;engagement&quot;, but it&#x27;s charged as that.

since you work in the industry - how about your opinion on the Uber case in the original post? that is PPC.

I work in digital marketing and this episode made me want to tear my ears off.First of all, they didn&#x27;t differentiate between display or PPC advertising. PPC you only pay if the user actually engaged with the ad. Nearly all of the anecdotes they used where about online display advertising - a known crock.And we absolutely run experiments all the time! In fact, we tie our ad spend directly to conversions. If anything, the market is too efficient - it&#x27;s really hard to get more than what you are paying for.

Did you read the transcript the for episode and the associated links and notes?They back up their claims with studies of their own as well as metanalysis.

This is really, really not true. Advertising lift has been well studied, especially in metastudies spanning hundreds of digital campaigns across Facebook and Google. These are independent academic metastudies, with hundreds of millions of impression data samples.Positive lift in the range of 0-20% is very common, and many statistical aspects of causal inference on ad impacts are well understood.Negative lift and flat campaigns are real phenomena too, and it does deserve more widespread publicity that negative lift happens in an appreciable number of campaigns, but that doesn’t take away from the overwhelming evidence that digital advertising works and that the mechanics of positive lift are well studied.Here are two of the foundational papers in this area:- <a href="https:&#x2F;&#x2F;www.kellogg.northwestern.edu&#x2F;faculty&#x2F;gordon_b&#x2F;files&#x2F;kellogg_fb_whitepaper.pdf" rel="nofollow">https:&#x2F;&#x2F;www.kellogg.northwestern.edu&#x2F;faculty&#x2F;gordon_b&#x2F;files&#x2F;...</a>- <a href="https:&#x2F;&#x2F;www8.gsb.columbia.edu&#x2F;media&#x2F;sites&#x2F;media&#x2F;files&#x2F;Garrett%20Johnson%20paper.pdf" rel="nofollow">https:&#x2F;&#x2F;www8.gsb.columbia.edu&#x2F;media&#x2F;sites&#x2F;media&#x2F;files&#x2F;Garret...</a>Particularly Figure 1 (page 26) in the second link. That figure alone utterly refutes any nonsense claim that digital advertising doesn’t have provably positive ROI.

The ad buyers aren&#x27;t smart enough to measure the actual effectiveness of their ads, and the ad sellers are not incentivized to teach them how to do it. This can go on for an arbitrarily long time.

I think the next major recession is going to be triggered by this bubble popping.

I get downvoted whenever I propose digital advertising bubble will pop once people wise up to the fraud.

Ecommerce sites have very fine grained measurement of their advertising spend and know exactly the ROI (which is why they focus so much on retargeting)

That&#x27;s comparing one type of ad or channel to another.But none of them test &quot;no ads&quot; vs &quot;ads&quot;.

This is completely false. Large advertising platforms have many A&#x2F;B tests that show significant differences in consumer behavior between groups that receive different ad treatments.Ads might be less efficient than some believe, but it&#x27;s super easy to see that they &quot;work&quot;, and advertising platforms do it constantly.

That means you have zero loyalty among your consumers. Don&#x27;t you see that as a problem?

This is nonsense. I had a startup completely powered by Google Ads. Ads brought in nearly 100% of the traffic. When my billing information with Google got mixed up and the ads stopped, the traffic went to 0 immediately.

So the movie was bad, and people didn&#x27;t go see it? Doesn&#x27;t seem related to advertising.Eminem released his last two albums without even announcing them. Word of mouth got them both to #1 on the charts.If you tried that same movie experiment with something like Avengers, I bet the results would look a lot closer to Kamikaze than Delgo.

&gt; It became the lowest earning wide release movie up to that point in time. Each screening averaged two people per screening.On the other hand it got displaced by Oogieloves which had $40 million in marketing costs. Critics found it mostly bad and the only award it won was for films produced in Brazil. Maybe there was a reason no one wanted to spend ad money on that movie?

Though not an intentional test, we found out what happens when a movie gets a wide release but does not advertise. In 2008, the movie &#x27;Delgo&#x27; was released on over 2000 screens with nearly no advertising. Because the production company could not find a distributor, they spent their ad money on renting the screens for a week, with the hope that some people would randomly see it and word of mouth would spread, leading to the theaters wanting to keep it for additional weeks. It became the lowest earning wide release movie up to that point in time. Each screening averaged two people per screening. More people saw Conan O&#x27;Brien making fun of the movie in his monologue than actually saw it in the theater.The reviews for the movie were poor but it had lots of household names as its voice talent, including Anne Bancroft in her final film. Good or bad, advertising likely would have lead to more people seeing it than two per screening. Of course there&#x27;s no way to know for sure how many more would have been enticed by the ads but we do know that going with zero advertising resulted in a huge disaster.

Maybe not for companies of uber’s size&#x2F;current reach, but small businesses definitely do benefit from ads. They see an immediate uptick in sales when they start advertising on various platforms.

Ads are somewhat like propaganda. There&#x27;s no proving it will work or not in short terms, but nobody can deny that persistent propaganda has a long-term effect on the whole population. You have think about a kid&#x2F;youth who listens to some propaganda for years when growing up.  If say you show an ad for a new Apple product. A small part of it is to inform consumers about it, the longer larger part is to enforce the Apple brand which has been happening over many years. You can&#x27;t say well it is not quite working so let&#x27;s stop doing Ads.

This article also sheds a lot of light on the whole issue of why people don&#x27;t realize when digital ads don&#x27;t work: <a href="https:&#x2F;&#x2F;thecorrespondent.com&#x2F;100&#x2F;the-new-dot-com-bubble-is-here-its-called-online-advertising&#x2F;13228924500-22d5fd24" rel="nofollow">https:&#x2F;&#x2F;thecorrespondent.com&#x2F;100&#x2F;the-new-dot-com-bubble-is-h...</a>

&gt; No one is willing to run the experiments necessary.This is incorrect.

Depending on what kind of advertising you are talking about. Direct response advertisers measure ads to the cents. And they know exactly what the ROI is and where the customer is coming from.

They do have data. It&#x27;s pretty easy to see sales without ads vs sales with ads and then do further testing to narrow down results from there. This has been done for over a century since the first billboards were put up.The amount of data generated by adtech today is staggering. The problem isn&#x27;t data or advertising, it&#x27;s the wrong people running the wrong campaigns for the wrong reasons.

They mention the common retort to this which is very bizarre to me: &quot;If online ads don&#x27;t work, then huge companies wouldn&#x27;t spend billions of dollars on them. Therefore they must work. You academics are just missing something.&quot;Ok, how does those companies know it works? They don&#x27;t have any real data to show that it does, just the fear that if they stop they&#x27;ll lose a lot of business.

&gt;No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior.This may be true but it&#x27;s a separate issue than the fraudulent clicks.Brand advertisement (the kind you are talking about, as opposed to closed-loop direct advertising) is an investment in a brand, that doesn&#x27;t get paid off in a day, or a week, or a month, or even a year. It adds fractions of a penny onto a customers value, every day from today into eternity. It&#x27;s an investment in your mind, but the long time horizon makes it, as you point out, nigh impossible to measure ROI.However, this is one of those cases where despite being immeasurable, it still works.Imagine doing a study on low-fat or low-fat diets and trying to measure health outcomes like lifespan, or heart disease, or cancer after just one month. You can&#x27;t do it. The best you can do is measure markers of these outcomes, like insulin resistance, blood triglycerides etc. Brand advertising is similar. You measure markers of long-term purchase intent. It&#x27;s not perfect.And just like actually doing a long-term study of diet for example is riddled with confounding variables and very hard (nigh impossible) to do well, so it is with advertising.

That&#x27;s great, thanks for your experience, but can&#x27;t comment on the argument provided?Isn&#x27;t it possible that these experiments you talk about are fatally flawed? I have serious doubts that a company can run and do well designed statistical experiments when academic experts are plagued by p-hacking and other foot guns.

&gt; without any attempt to validate their investments.I&#x27;ve also worked in adtech, for quite a long time on all sides, and my experience is that far more companies do research to justify their ad spend not validate it.I have managed plenty of A&#x2F;B tests in my day, each one claiming to show some improvement, 10%, 15% etc (some, as you said, showing no improvement). Even though these tests are often run &quot;correctly&quot;, essentially nobody goes back and asks &quot;wait, we had a 10% increase multiple times in the past year, but is our &lt;metric&gt; really showing the cumulative improvement we expected?&quot;The greatest trick in the industry is that because VC are pouring money into everything, everywhere, every metric appears to grow. At every startup, everywhere, pre-pandemic, numbers were going up because people were pouring money into the system.I&#x27;ve worked at companies who I know for a fact their adtech product cannot and does not work, yet their business continues to explode because in recent months nearly all ad spend has been on digital ads.I&#x27;ve talked to companies whose entire function is bidding optimization who literally do not understand how to optimize bidding given the information you have.Absolutely people are running &quot;experiments&quot; but the function of the &quot;experiments&quot; is to justify that the ad team is worth having, and then that the VP of marketing is doing their job and then that the CEO has hired some super smart people, and then that the VCs might have really found a unicorn. Everyone sees what they want and no one really wants to ask the question &quot;wait, does this really work? are these tests really able to capture the complexity of the environment?&quot; And if you are one of those ornery people that insists on probing into the details and seeing if any of this is working, you will eventually get fired.Ad tech is largely a scam, but a huge number of rich and smart people benefit from the illusion that it is not, so we continue to see experiments showing that everything works as expected.

&gt; I work in adtech, and almost all companies run experiments in order to optimize their ad spend ... It&#x27;s incredibly naïve to think that companies are flushing half a trillion dollars a year down the toilet on advertising without any attempt to validate their investments.We have a very large and conspicuous example here that suggests that Uber hasn&#x27;t been doing this.Running experiments like you suggest is difficult and to do them right requires some fairly specialized knowledge to do correctly. I&#x27;d doubt more than a small percentage of companies have the expertise to effectively run any kind of advertising experiment that returns useful data. Most businesses lack that knowledge and rely on metrics provided by the people selling the advertising—who are likely to provide self-serving numbers.While I&#x27;m sure some of the bigger F500 companies do a good job validating their ad spend, I most companies don&#x27;t even know how. Certainly the majority of small businesses have no idea how effective their advertising spend is aside from very crude word-of-mouth feedback. I suspect this creeps way up into the Fortune 500 as well.

&gt; No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary.This is entirely false. I work in adtech, and almost all companies run experiments in order to optimize their ad spend (everything from Ad A vs Ad B, to Ad vs No Ad, to Channel A vs Channel B, and more).This isn&#x27;t to say that advertising always produces ROI. Quite often, experiments will be run that will show a certain strategy isn&#x27;t performing well, and the company will adjust accordingly. It&#x27;s incredibly naïve to think that companies are flushing half a trillion dollars a year down the toilet on advertising without any attempt to validate their investments.

&gt; No one is willingCounter-example: I know of at least one consumer goods company that has studied the long-term effects of certain kinds of sponsorship deals on consumer behavior. The study had tracked people for at least 10 years at the time I learned of it. Of course, this company would never publish the results, because they provide a competitive advantage in structuring and bidding on sponsorship deals.

You are conflating  value of the marginal ad with value of any ads at all.

You are conflating value of the marginal ad with value of any ads at all, and also exaggerating what those articles sayOne of them said tracking cookies only boosted conversion 4%, and another said P&amp;G did better with some traditional media advertising than some digital advertising.

&gt; No one is willing to run the experiments necessary.Because advertising is like military spending. It takes a lot of money to maintain the status quo.

This is patently false. I have friends who have worked years developing solutions for doing ad effectiveness comparisons. They are A&#x2F;B tests in most cases but some methodologies require a lot of sophistication because of problems tracking conversions.Traditional ad platforms like TV, newspapers might not have done this but online ones surely do. Infact that is one point they consider an advantage as you can measure effectiveness unlike the traditional platforms.

I think they did the A&#x2F;B testing by switching off 100m of ads and having the same results.

This is blatantly false. It’s incredibly easy to prove that digital advertising is effective using A&#x2F;B testing.

until you realize that ads are meant to make FB and Google rich, not you, you&#x27;re going to have a bad quarter.

The implication here is that money spent on ads is largely wasted.Companies like Uber and Ebay have turned off all their adspend and saw little to no change in their acquisition metrics. You can argue that they were just doing it wrong. But the point is that, if even they are doing it wrong - and getting nothing in return for the millions they&#x27;re spending on ads - then it&#x27;s very likely most others are in the same situation.You are right that this doesn&#x27;t mean _all_ advertising is useless, there are absolutely profitable usecases. But the larger points still stands: most money being spend on advertising right now is likely not returning anything.We now have some strong precedents being set. I believe this will cause more major companies to run the ultimate experiment: turn off all ads and see what happens. It&#x27;s too early to tell, but it&#x27;s not impossible we&#x27;ll see adspend drop significantly across the industry once everyone finds out they&#x27;re just burning money.

This was a superficial analysis at best. Adtech produces petabytes of data every day proving that advertising works. There&#x27;s a reason why two of the most valuable internet companies sell ads.The problem is knowing exactly which formats and campaigns are working down to the dollar, but part of that is just the reality of fuzzy attribution and it&#x27;s only getting harder as privacy regulations get stronger. However you can definitely tell the difference when turning everything off, and if you can&#x27;t then you were advertising to the wrong people in the first place.Uber&#x27;s mistake isn&#x27;t that advertising didn&#x27;t work, but rather that they didn&#x27;t vet their vendors or even bother doing any checking and optimization of their own.

At least in e-commerce it’s very much possible. The company that I work for does such experiments regularly (there’s a dedicated Data Science team for measurement) and I’ve personally been involved in lift studies for Google Ads. They work, you just have to be careful with the ‘how much’ combined with ‘for what’.Happy to chat with anyone who is interested in the topic (pfalke at pfalke dot com).Haven’t had a chance to listen to the podcast, apologies if that made me miss the point of the parent post!

It&#x27;s an arms race. In theory advertising should give you an edge over non-advertising competitors, but if everyone is doing it demand remains unchanged and also you&#x27;re wasting money

Lewis and Rao published a meta-analysis of 25 large scale controlled advertising experiment [1]. Here&#x27;s the abstract:Twenty-five large field experiments with major U.S. retailers and brokerages, most reaching millions of customers and collectively representing $2.8 million in digital advertising expenditure, reveal that measuring the returns to advertising is difficult. The median confidence interval on return on investment is over 100 percentage points wide. Detailed sales data show that relative to the per capita cost of the advertising, individual-level sales are very volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than 10 million person-weeks, making experiments costly and potentially infeasible for many firms. Despite these unfavorable economics, randomized control trials represent progress by injecting new, unbiased information into the market. The inference challenges revealed in the field experiments also show that selection bias, due to the targeted nature of advertising, is a crippling concern for widely employed observational methods.[1] <a href="https:&#x2F;&#x2F;academic.oup.com&#x2F;qje&#x2F;article-abstract&#x2F;130&#x2F;4&#x2F;1941&#x2F;1914592" rel="nofollow">https:&#x2F;&#x2F;academic.oup.com&#x2F;qje&#x2F;article-abstract&#x2F;130&#x2F;4&#x2F;1941&#x2F;191...</a>

There was a freakonomics podcast recently about advertising (online and traditional).No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior.<a href="https:&#x2F;&#x2F;freakonomics.com&#x2F;podcast&#x2F;advertising-part-1&#x2F;" rel="nofollow">https:&#x2F;&#x2F;freakonomics.com&#x2F;podcast&#x2F;advertising-part-1&#x2F;</a><a href="https:&#x2F;&#x2F;freakonomics.com&#x2F;podcast&#x2F;advertising-part-2&#x2F;" rel="nofollow">https:&#x2F;&#x2F;freakonomics.com&#x2F;podcast&#x2F;advertising-part-2&#x2F;</a>

But that would still make their ad spend useful. Even if not in the traditional marketing sense.But if you can&#x27;t see any change in business from 2&#x2F;3 of ad spend at all it must be fraudulent.

I would guess it&#x27;s fairly hard to encode a lot of rules that are obvious to most people algorithmically. Amazon does encode a likelihood of repeat purchases to some degree; they offer a coffee subscription but not a weed whacker subscription.But besides doing something like &quot;purchases &gt;$100 are not likely to be repeat purchases,&quot; I would think it would be difficult.

The answer to your confusion is that un sophisticated remarketing (showing ads for searched or purchased items) has in nearly all cases a measurable improvement on ROI. Improvements to filter out useless ad views are much more complicated and expensive. 
Advertisers actually don’t care about any individual case, but buy on aggregate results.

Agreed. Companies often don&#x27;t understand the &quot;directionality&quot; of purchasing behavior.For instance, I recently bought a new iPhone directly from Apple, and I bought a case for it on Amazon. Now two months later, Amazon is recommending that I buy an iPhone (...which I bought two months ago from Apple).Buying a case an then waiting two months to buy the actual phone would be a very strange thing to do, but Amazon seems to think I should do this.

&gt;In short I am not really convinced that these platforms are good at predicting what I want. Rather it looks as if they are good at showing me what I already have.While that&#x27;s definitely true (I get ads from Amazon for stuff I&#x27;ve already purchased all the time too), in that case Amazon is leveraging their own marketing channels to support their own business -- which has, effectively, zero marginal cost to them.However, when advertisers buy space from Google or FB, that&#x27;s not the case. There is a definite external cost to such ad buys.As such, being able to identify the &quot;value&quot; of such expenditures is (or can&#x2F;should be) important to the advertisers.That said, much of advertising is based upon the idea of &quot;top-of-mind awareness.&quot;[0]The idea being that if you are considering a purchase, the brand that comes to mind without prompting when considering that purchase will be preferred over brands of which you aren&#x27;t immediately aware.Purchase decisions made on a whim or without any research are usually those which are low cost. Which is why companies like P&amp;G, Coca Cola and like companies focus on &quot;top-of-mind awareness.&quot;If you&#x27;re going to buy a washing machine or a riding lawn mower, you&#x27;re much more likely to do research than if you&#x27;re going to buy a soft drink or a bag of chips.That said, the same &quot;top-of-mind awareness&quot; can be helpful even for big ticket items, as those brands may well be the first ones about which research may be done.Identifying how such advertising may impact purchase behavior is a complex topic, and unless you can quantify specific ad-views to actual purchases, doing so is generally speculative and is the subject of a great deal of quantitative market research[1].[0] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Top-of-mind_awareness" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Top-of-mind_awareness</a>[1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Quantitative_marketing_research" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Quantitative_marketing_researc...</a>

The probability that a random person would want to buy a blade for a bread cutting machine is nearly 0%. The probability that you will buy a blade for a bread cutting machine again is non-zero because you may be dissatisfied with your new one.

&gt; Yeah I think Google and other advertisers are great at predicting what people will buy and then shows them ads for it.Seems to be entirely subjective.I bought a new blade for a bread cutting machine on amazon, the last one lasted ~25 years. My current recommendations: blades for bread cutting machines.I also checked some Christmas decorations on Amazon, but didn&#x27;t buy any. So next to the blades I get Christmas decorations recommended. January is a bit late for that, isn&#x27;t it?I also bought a single server rack on eBay. Now I regularly get spam emails from eBay reminding me of new offers for server racks. Even the company I work for doesn&#x27;t have that many, so why?In short I am not really convinced that these platforms are good at predicting what I want. Rather it looks as if they are good at showing me what I already have.

Yeah I think Google and other advertisers are great at predicting what people will buy and then shows them ads for it. That’s different than showing a person an ad and changing their behavior, but to advertisers they can’t know unless they experiment.

So programmatic means using a 3rd party ad tech tool to place your ads across all internet ad placements.Doing it directly means placing ads yourself on the biggest platforms (Facebook, google).Direct response just means &quot;if you actually want results&#x2F;ROI&quot;, as opposed to for nebulous brand goals.

Care to elaborate more? What exactly is &quot;programmatic direct response advertising&quot;?

Friends don&#x27;t let friends do programmatic direct response advertising. Do it directly on the platforms yourself.I run an ad agency with relatively well known clients and I consider the word &quot;programmatic&quot; in a resume a negative correlation.

I&#x27;ve said for a long time that Martech is due for a correction due to lack of understanding in attribution.  Could this finally be the bubble bursting?

I&#x27;m reminded of the famous quote by George Best: “I spent a lot of money on booze, birds, and fast cars. The rest I just squandered.”

It&#x27;s also an important signal to consumers.

I&#x27;m genuinely shocked at how many people think advertising doesn&#x27;t work.Just because it doesn&#x27;t make a person consciously stand up and say, &quot;I want a Snickers&quot; does not mean the ad didn&#x27;t work.Like, honestly, are there people here suggesting viral ads don&#x27;t work? Or don&#x27;t understand the point of Coke or Mercedes Benz ads are for? Coke establishes themselves as the defacto soda. There is no other soda, or if they are, wish they were coke.MB ads among other things reinforce to MB owners the wisdom and luxury of their brand. To own an MB is to be part of a club, one that is advertised across the media spectrum. If MB didn&#x27;t advertise at all, they would either become another commodity brand, or have to be so luxury that only word of mouth is necessary (Bently, etc).Now, maybe that&#x27;s what cortesoft is saying &quot;get you to waste money&quot; but if it got you to waste money on their product, it worked.

I&#x27;m not entirely sure about that, at least not in all cases.I was recently in the market for a new mattress, so went to a mattress store and tried some out. Even though Tempur Pedic is all the rage with their insane ad spending, I was not all that impressed, and found a considerably cheaper mattress that felt much higher quality.Now, this may all be sales mumbo-jumbo, but the sales rep noted that a lot of people come there only interested in Tempur Pedic mattresses, which he regards as &quot;literally a piece of foam, total crap mattresses.&quot; They sell at such a high price point because they advertise so much, while the mattress I found came from a no-name brand that doesn&#x27;t advertise at all, and was therefore cheaper.So I guess in some markets, advertising does work. I&#x27;ve never heard of someone telling me how much they loved a mattress unless it&#x27;s a Sleep Number or a Tempur Pedic, and those advertise by far the most.

This was true in the past, but nowdays it’s much easier to try out a product and see reviews.With Uber it would be so easy to penalize drivers who cancel because they don’t like where I’m going, but I’m still wasting 30 minutes quite often, because Uber spends that money on ads and UI rewrites instead of improving the core experience.

The point of traditional advertising IS pretty much to waste money, similar to how the point of a peacock&#x27;s tail feathers are to waste resources... it is a signal to potential partners (business or romantic) that they are so capable they can waste resources. It is a signal of bonafides.

this confirms the previous post if you subscribe to the idea that the purpose of advertising is to waste oxygen

I suppose this is true in the same sense that the purpose of life is to waste oxygen.

I think a lot of this was already obvious to slightly-savvy mobile app users. Look at pretty much any free-with-ads app; a majority of the ads are going to be ads for other free-with-ads apps, many of which you already have installed. The thing is, outside of the Internet most ad spend is for marketing ubiquity, not direct response. You don&#x27;t really buy, say, TV advertising with the expectation of getting so many clicks or calls out of it. And, up until a few years ago, TV was the lion&#x27;s share of ad spend. So who knows if we&#x27;ll see an actual industry reckoning or not. The whole point of advertising is to waste money, after all, and plenty of brands were fine with getting nothing but exposure out of it.More interesting to me is the fact that tech companies are finding it surprisingly difficult to control where their ad-spend goes. I suppose this is an inverse of the problems with supply chains, where Apple can take three years to get a connector vendor out of their supply chain even when they were using literal child&#x2F;slave labor. It&#x27;s a market for lemons; bad &quot;money&quot; (publishers, suppliers) drives out good. The question is: will questionable ad publishing actually harm corporate reputation to the point where big ad spenders go away, or will we just see periodic Adpocalpse-style waves of spending being decreased and then brought back?

I guess there are 3 reasons why reducing ad spend might not show a negative effect:(1) Due to fraud, the money wasn&#x27;t actually going to ads.(2) You didn&#x27;t need the ads. You were over some saturation threshold and the law of diminishing returns kicked in hard.(3) The ads were valuable to you but not in an easily measured way. Maybe you could coast on brand awareness inertia for 6 months or a year before it affects sales.This case seems to have decent evidence for at least part of it being #1, but not necessarily all of it.

you are right; some of the biggest brands need to do the least advertising

Playing devils advocate, couldn’t this just be reflective of the fact that Ubers brand reach us strong enough that there is very little advertising that gives them a marginal benefit?Uber owns taxi services as a brand as much as Google does search.

I like that this story references the #deleteUber tag on Twitter because it helps organize the timeline of events going on at Uber.There was another twitter thread on here recently about the chaotic and disastrous (though successful) Swift rewrite they did which happened during the same time frame. <a href="https:&#x2F;&#x2F;twitter.com&#x2F;StanTwinB&#x2F;status&#x2F;1336890442768547845" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;StanTwinB&#x2F;status&#x2F;1336890442768547845</a>During those time frames is also when all of the execs were quitting or being fired, along with their internal harassment problems.It feels like Uber has really succeeded in spite of itself.

First signs of bubbles are fraud and wrong valuation.

Maybe. A lot of people have been saying this for over 10 years. If it&#x27;s a bubble that&#x27;s real though, the effects are going to be pretty widespread. Not only do two of the biggest (and best-paying) tech employers get slammed, but so do all the inter-related companies. And, also BTW there will also be a lot fewer fat exits for startups if Google and Facebook acquisitions get turned off.

It&#x27;s critically important for adtech companies&#x27; customers, since it&#x27;s an essential part of determining return on investment.But it&#x27;s also critically important for both adtech companies and data scientists who work in the space to direct people&#x27;s attention away from those sorts of metrics. You generally don&#x27;t want to call the attention of the person who signs your paycheck toward the fact that it&#x27;s all but impossible to really know for sure if your service has delivered them any net benefit.

Does advertising create an increase in buying your product. That’s the most important question.

There&#x27;s still value in getting people to buy now instead of maybe buying later.

The thing about the &quot;advertising doesn&#x27;t matter&quot; argument is that there are situations where advertising absolutely matters - when a company is just starting out and there&#x27;s no way they can get organic referrals, when the public is unaware of their existence, etc.Moreover, most companies started small at some point so advertising and maintaining advertising made sense up to a point.And if we look at those companies where maybe advertising actually doesn&#x27;t help, companies that have reached a level of success where organic referrals and public awareness drive most of their business. And they&#x27;re ongoing businesses that probably reached that level through advertising and have money now to use for that.But not only are they already advertising but they don&#x27;t if their word-of-mouth&#x2F;public-knowledge presence will last indefinitely, they don&#x27;t know if just word-of-mouth would let them control their image, would stand-up against future competitors ads and so-forth. So, even supposing you could show advertising didn&#x27;t offer any immediate increase in customers, continuing to spend on it doesn&#x27;t seem to me as irrational as it sounds.

Maybe the person is referencing how difficult it is to determine the value of the counterfactual? If it were possible to determine I imagine people would really want to know.

I think the implication here is &quot;that honestly doesn&#x27;t seem that important&quot; because the expectation is that the number of people who would find you anyway would be a small number compared to a well-run ad campaign, at least over timeframes startups care about.Having worked with a decent number of small teams without a lot of brand recognition, ads bring in a lot of customers: it was pretty obvious in our metrics when we ran ad campaigns vs when we didn&#x27;t. Sure, eventually a few of those people may have found their way to us somehow anyway, but there was a step change in volume from ads. At least in my experience, the idea that ads have no significant impact is not really supported by the evidence. I assume that&#x27;s what the OP meant by it not seeming important: they&#x27;re working under the assumption that &quot;do any ads do anything&quot; is clear and you don&#x27;t need to prove the worth there — instead you need to prove which ads work better than others, and cull the ones that don&#x27;t perform well (which will also help save you from scammers).One could argue that advertising for large brands with name recognition could be less valuable. I don&#x27;t really have data or experience to know that one way or another, but in a competitive market that seems a little hard to believe. Tesla is an interesting counterexample, but I think it may be in part due to the EV market in the US not being particularly competitive; no one makes attractive, mid-priced cars with long range other than Tesla, let alone having an established charging network for road trips. And Elon&#x27;s antics are, tbh, a form of marketing in and of themselves; &quot;all press is good press.&quot;Edit: I wrote the above paragraph badly. I can definitely believe ads are less valuable if you&#x27;re a big brand vs a small one. But I&#x27;d be surprised if they&#x27;re useless, at least in a competitive market. People don&#x27;t have infinite money, so if they&#x27;re looking to buy X and an ad for X from your competitor pops up, it seems reasonable to me that some percentage of the time your competitor would make the sale and thus you wouldn&#x27;t, even if organically that sale might&#x27;ve gone to your company instead.Also, I wouldn&#x27;t argue that even small companies need ads — just that they do seem to be effective at increasing the volume of people interested in using your product at a given time. That may or may not be necessary&#x2F;useful depending on your situation. I have been on teams where we intentionally turned off ad campaigns because we learned what we needed to learn from the cohort of new users, and now wanted to improve the product based on their feedback before spending money on more ads.

&gt; Sure, you&#x27;re never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it. But that honestly doesn&#x27;t seem that important compared to the other metrics...Wait, why wouldn&#x27;t that be important? It seems like the most important question since it asks whether advertising has any significant impact at all.

There’s an interesting question there about when this form of advertising becomes obsolete.Uber is in a very different position than many other companies. Anyone who browses the internet with regularity is already aware of their existence and probably just needs the right set of circumstances to come together to make Uber useful to them.I suspect the results would be different if Uber were earlier in their adoption curve, but maybe that’s not true either. Maybe they’d be ignored for different reasons at that time.

Paying for crappy app installs on shady sites is not really advertising.What they should have done was a proper hyperlocal SEO campaign like Firestone Gieco, and Mc Donalds do.

There&#x27;s a huge difference between a company that is widely known for being the app for ride hailing and a small company trying to get eyes on their new product.

Well, Uber is known to everybody by now. To the effect that &quot;to Uber&quot; is even kind of a verb&#x2F;noun.Also people either need a hired transportation or not. If they don&#x27;t have a car or don&#x27;t want to mess with the traffic and need to go somewhere, it&#x27;s either Uber or Taxi usually.It&#x27;s not like Coca Cola, which is well known, but people could do without it (unless addicted), so needs to constantly nag people.And it&#x27;s not like some new product, which without advertising nobody would even know it existed.In fact most of Uber&#x27;s existance its operation has been 100% advertising (spending VS money to offer cheap rides and expand and gather &quot;eyeballs&quot; and &quot;customers&quot; without a profit). In my book, customer acquisition without profit is another name for advertising.So it doesn&#x27;t sound strange that it could do without advertising today.But what if there were 3-4 strong players in the same, each eating in Uber&#x27;s market share? You&#x27;ll see how fast they&#x27;d found advertising indispensable again...

Literally lighting them on fire in a cold office to help provide some BTUs would’ve been a better use of that money, since at least that would offset spending somewhere else.

Did you actually read the linked tweets?!Uber turns off a shitload of ad spending, nothing bad happens to new user acquisitions.I&#x27;d say there are about a million better uses of a million dollars than just pissing them away on a scam.

It would work for some of Uber&#x27;s ads. From the article, it looks like they were frauded mostly by in-app ads. And they were paying for installs - not actual trips.So, no, Uber&#x27;s advertising here is a little different than (I think) the majority of companies. They are mostly paying for installs rather than sales &#x2F; conversions. A lot of newer &quot;app&quot; companies could be in similar situations.Though, honestly, this seems like a massive fail on their analytics team for not figuring this out earlier. They should have been able to see that all of these &quot;installs&quot; from certain advertisers were not leading to trips.In fact, it says they turned off 66% of ads. They didn&#x27;t randomly turn of 66% of ALL ads. They turned off this TYPE of ad, which they failed to earlier recognize was ineffective.Step 1) assume your ads won&#x27;t work.Step 2) have enough analytics &#x2F; logging in place to convince yourself the ads do work.Step 3) if they don&#x27;t work, turn them off.Looks like they skipped step 2 - which honestly, is not uncommon for a fast growing business - even if they are huge and already make a lot of money.What they found isn&#x27;t even what people are discussing. They found that certain networks they were buying ads from were almost 100% fraud (which is pretty well known).Instead, people here seem to be discussing that most online advertising is fraud, and&#x2F;or that there&#x27;s no way to prove it&#x27;s effective. That is absurd.

&gt; All big companies do this.Except Uber, apparently? Or would the method you&#x27;re talking about not have discovered that something fishy was going on? (I don&#x27;t work with ads so I don&#x27;t know the limitations of the type of experiment that you&#x27;re talking about)

&gt; You can A&#x2F;B test ads pretty easilyA&#x2F;B testing ads is a complex matter. you can A&#x2F;B traffic and conversion easily, but a lot of established companies with fierce competition fight for mind share, not direct conversion; for that, you have both awareness effects (user won&#x27;t forget about coca cola if they don&#x27;t run ads for a month, and an ad that doesn&#x27;t directly convert but increase awareness still has value) and coverage synergies (the number of repetitions in a day will increase coverage non linearly and the amount of channel repetitions will increase awareness more than a single channel view, even if it doesn&#x27;t convert immediately)

Instagram definitely does this. I&#x27;m part of the magic cohort who never see any ads :)

&gt; Sure, you&#x27;re never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it.This is typically an order of magnitude less than the attribution figures stated by digital marketing experts. A&#x2F;B test is the right method to use, but the crucial thing is you need an earmarked population to see zero adds over your attribution window, since what you care about is impact on incremental sales, rather than incremental click likelihood.There is a long econometrics literature on this and it is not a fussy technicality, the figures typically differ by 10x +.

It can happen that an almost certainly bogus scientific field persists for decades using state of the art research methods and no bad faith on the part of the researchers (any bias they introduce is probably not conscious). <a href="https:&#x2F;&#x2F;slatestarcodex.com&#x2F;2014&#x2F;04&#x2F;28&#x2F;the-control-group-is-out-of-control&#x2F;" rel="nofollow">https:&#x2F;&#x2F;slatestarcodex.com&#x2F;2014&#x2F;04&#x2F;28&#x2F;the-control-group-is-o...</a>A&#x2F;B testing is... not a state of the art scientific research technique. Moreover the companies that provide the tools to do A&#x2F;B testing are the same companies that sell you access to advertising space. I&#x27;m not saying that it&#x27;s a common practice to defraud A&#x2F;B tests, I&#x27;m saying that the fact that adtech companies have chosen to enable that research methodology out of the set of all methodologies they could offer suggests that we should expect, before seeing the results of any A&#x2F;B trial, that the results will tend to favor the adtech narrative.I don&#x27;t think that people in the adtech space believe they&#x27;re selling a bogus product, but I do think they wouldn&#x27;t want to know if they were — they have a good thing going.If you&#x27;re employed in adtech you&#x27;re mostly fine, skills transfer. If you&#x27;re invested in adtech, do what you can to diversify away. Advertising is overvalued to some extent and that bubble will burst at some point or other. The question is just how much actual value advertising provides, how much will remain when the bubble bursts.

&gt;but you can also measure against SEO traffic with a high degree of certainty.None of these benchmarks distinguish between the selection effect (clicks, purchases and downloads that are happening anyway) and the advertising effect (clicks, purchases and downloads that would not have happened without ads).You can fix this by dividing the target group into two random cohorts in advance: one group sees the ad, the other does not. Designing the experiment thus excludes the effects of selection.When you do this experiment correctly, you find out that ads have low effect or are not cost effective (as eBay discovered).

So how much ad spending is going to &#x27;dubious ad networks&#x27;?Are they easy to categorize? Or is it a big secret what&#x27;s dubious and what isn&#x27;t?

This is all discussed in the podcast.There’s no revenue change even after testing (when you tack on costs of ad delivery).

For native Android you 100% have access to an Android &quot;device id&quot;. I set this up on an Android app we delivered. There&#x27;s even companies that pay other Android apps to host their code inside the app, co-related that ID to other data available to the web (e.g. IP address, installed fonts, etc) to provide a very good guess of a user&#x27;s device id via the web.Apple had something similar, but you&#x27;re right they&#x27;re starting to turn it off. In my past life in Adtech we noticed the click through rate of users with Android user agents were a lot higher on certain campaigns than iOS users for this reason.

&quot;You can A&#x2F;B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another.&quot;Nope.You need a unique id for an AB test AND apple wont give this to you. No company is going to give you a click&#x2F;tap id. Android might. This is so inaccurate I don&#x27;t know where to start. Apple has turned off all unique ids that work across an AB test [1] cross platform. You would need on the ground mobile ad experience to know this.<a href="https:&#x2F;&#x2F;www.invoca.com&#x2F;blog&#x2F;what-is-idfa-and-why-apple-killed-it-everything-marketers-need-to-know" rel="nofollow">https:&#x2F;&#x2F;www.invoca.com&#x2F;blog&#x2F;what-is-idfa-and-why-apple-kille...</a>

Across this and the other[0] highly active digital ads post, there&#x27;s 3 interesting forces at play:a) technologists screaming &quot;ads are literally the worst societal cost, like ever&quot; who don&#x27;t want to understand the industryb) advertising folks taking up arms to defend their shamanistic, money-printing machines without statistics (Reason No. 7 will SHOCK you)c) A tiny, tiny group of Mandalorian-like voices who have a necessary statistics AND industry understanding who are being drowned out(this is in jest, but I double-dare you to say it isn&#x27;t at least directionally accurate)Other commenters have mentioned it throughout this whole comment thread as &quot;incrementality tests&quot;. Amongst other approaches, this is the way.Freakonomics severely lack the industry understanding. Listening to the podcast was like hearing how HTML is a programming language from the kid in week 2 at code camp. Then too the article, all the issues with Uber was just doing a bad job of managing their ad spend and they can fall into group B, noted above.The baseline of this work is a control group who see no ads and then you build your tests from there that factor in channels, cohorting, and other components to get a statistically significant outcome. Yes, this will get more challenging with upcoming privacy changes (IDFA removal, et al). However, the last 10 years this wasn&#x27;t a problem and I&#x27;m sure the corporations in the identity resolution business will hand-shake on a bunch of 2nd party data deals that just move the deals done in broad daylight around identity tracking and audience creation to the alley. Further, any advertising that is tied to an already known customer is able to be backed into at an audience level with login and cookie data. Even Pi-hole users may not be exempt here.To finish with some constructive advice:1) Advertising is not a synonym for marketing. We&#x27;re only talking about advertising in both HN threads.2) Every industry has high and low quality. Pareto&#x27;s principle should be aggressively applied to where one spends their budget in the cesspool that is the Internet.3) If you&#x27;re ever seeking an agency to provide advertising services and they don&#x27;t have a qualified data science or statistics leader (10+ years work experience, degree in Stats&#x2F;Math&#x2F;Econ, an MBA, or similar), run. Run from those shiny-shoes gurus. Channel your inner Usain Bolt and run.0. <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=25620707" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=25620707</a>

You can definitely measure &#x27;lift&#x27; - though it&#x27;s far easier with online only sales.For sure it&#x27;s harder for say Gucci to measure incremental cross-platform lift but any good large brand spends big money trying to parse it outOne example way to get some value measurement would be Facebook&#x27;s powerful tools. Beyond a basic 1:1 audience 50&#x2F;50 exposure test, Facebook also lets you upload offline sales (or use their pretty effective pixel for real time online sales) and then feed into their system. Using the data your purchase data FB matches to ads delivered and you (with PII but also purchase value and frequency). Then the advertiser can define lookback&#x2F;conversion definitions to get a measure of incremental lift. E.g. those who clicked ad bought __ right away, those who viewed ad within 1, 7 or 28 days bought __.For what it&#x27;s worth the adtech forums I participate on view Uber as incompetent idiots on this issue (which has been reported on like last year) - all they had to do was measure beyond the initial install for say did these installs actually pay to ride...Install fraud from SSPs and resellers is huge, doing this basic measurement and quality control is the reason Facebook and Apple have huge and growing app install business. Though TBD on Apple fucking over Facebook and keeping the attribution for themselves alone with their new privacy features

You can A&#x2F;B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another.You don&#x27;t have control over your SEO results as well - but you can also measure against SEO traffic with a high degree of certainty.All big companies do this.Sure, you&#x27;re never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it.But that honestly doesn&#x27;t seem that important compared to the other metrics - which most functioning large companies have decent data on.You&#x27;re also never going to know how many of your customers ate Green Eggs and Ham for breakfast. It&#x27;s irrelevant. You have decent insight into your direct-online ROAS, and that&#x27;s unique to direct online advertising, and it&#x27;s important!!Yes, you&#x27;re never going to know if that million dollars you invested in online ads was the best use of that million dollars. But that&#x27;s not much different than building a new factory, either.Edit:Specifically to Uber&#x27;s case - they did NOT turn off 66% of ALL ads randomly to no adverse effect (implying that all ads are worthless).They DISCOVERED that a certain type of ad (paying for installs on dubious ad networks) was mostly fraud. After turning off 100% of this type of ad - they found no adverse effect.This is a fail on their analytics team. They should&#x27;ve been measuring this type of ad better - especially given how big a portion of the total spend it was - and had insight into something not being right. They should have been able to do this - and if they couldn&#x27;t, because the network somehow didn&#x27;t give them enough data to do it, they probably shouldn&#x27;t have been spending this much money for exactly these reasons!

I read a news article yesterday that had the same ad repeated roughly 20 times. Would the metrics show a separate impression for every one of those ads I scrolled past?Also, despite the repetition, I don&#x27;t remember what it was for. I think it had a photo of a woman outdoors.

Might be a possibility to disrupt the ad business. FB and Google might be paid tons of money for providing little value or maybe even toxic value.

Uber discovered they’d been defrauded out of 2/3 of their ad spend