... and the HN link&#x2F;discussion (submitted by jgrahamc): <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=13718720" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=13718720</a>

&quot;enable AMP, and more&quot; is a certainly nicer than &quot;&quot;ScrapeShield&quot; feature which parses and obfuscates html&quot;.

&gt; Incident report on memory leak caused by Cloudflare parser bugThis title sounds like Cloudflare doesn&#x27;t know what a memory leak is or are intentionally trying to downplay information disclosure. Neither option is comforting.

Not the TLS Private key, this would pertain to the ClientKeyExchange. The TLS Private Key, should NEVER leave the server. The buffer overruns was only what a client&#x2F;server exchange would see.

From the blog post: &quot;For the avoidance of doubt, Cloudflare customer SSL private keys were not leaked. Cloudflare has always terminated SSL connections through an isolated instance of NGINX that was not affected by this bug.&quot;Is this statement accurate considering Tavis said in his report that: &quot;We fetched a few live samples, and we observed encryption keys, cookies, passwords, chunks of POST data and even HTTPS requests for other major cloudflare-hosted sites from other users.&quot;

&gt; MITMing all CloudFlare sites with a warning for users to change their passwordsREALLY?

Why is your company severely downplaying it?Honestly, this is the biggest security incident in a long time, and proper mitigation would probably warrant:- forcefully terminating all cookies on CloudFlare sites, cloudflare already injects JS onto the page anyway- MITMing all CloudFlare sites with a warning for users to change their passwords

Full details from Cloudflare: <a href="https:&#x2F;&#x2F;blog.cloudflare.com&#x2F;incident-report-on-memory-leak-caused-by-cloudflare-parser-bug&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.cloudflare.com&#x2F;incident-report-on-memory-leak-c...</a>

That was my tweet believe it or not. I had to turn notifications off on my phone because out of nowhere it was getting bombarded with shares&#x2F;likes...

I remember Tavis tweeted Friday night asking for a cloudflare engineer to contact him, and everyone joked that the last thing you want on a Friday evening is an urgent message from tavis ormandy.

Nah. Bug bounties don&#x27;t work for services like CDNs. Maybe they do elsewhere. But for enterprise services, the noise rate is too high, and the very good bug finders are either salaried, free, or working for the adversary.

The reward includes a t-shirt, it isn&#x27;t a mere t-shirt. You also get &quot;12 months of CloudFlare&#x27;s Pro or 1 month of Business service on us&quot; (~$200). The reward is also not tiered.The award may still not be all that much, but let&#x27;s not make things up about them.

What would make sense (to me, not a business&#x2F;marketing guy, nor a lawyer, at all) would be a t-shirt and free subscription as the offered thing, something which costs the company nothing.Then for anything like this, give publically a bonus gift which makes it worth people reporting to them and not blackmarket selling it. Once it&#x27;s gone through the legal dept. and so on.Then they can be very quick with handing out tshirts and so on to any and every microissue report, without the people running triage having to care about amounts or tax or whatever.Having any kind of publically offered payment for service (beyond a tshirt bounty or services in kind) is just begging for legal issues, right?

I would say the crazy thing is a mere t-shirt as their &quot;bug bounty&quot; top tier award given how they&#x27;ve pitched themselves as an extremely secure service.<a href="https:&#x2F;&#x2F;hackerone.com&#x2F;cloudflare" rel="nofollow">https:&#x2F;&#x2F;hackerone.com&#x2F;cloudflare</a>I&#x27;m sorry but when the reward for breaking into you is basically a massive pinata of personal information...that simply is a bad joke. Security flaws are going to happen and if you aren&#x27;t going to even offer a reasonable financial reward to report them to you, well, that is just begging to be exploited with a pinata that size.

It shouldn&#x27;t be too difficult to feed an instrumented copy of the parser some fraction of their cached pages (after all, that&#x27;s what they&#x27;re for.. right?) and calculate a percentage of how many triggered e.g. valgrind, or just some magic string tacked on the end of the input appearing in the output or similarI prefer CloudScare to Cloudbleed :)

You really want to see Cloudflare spend more time discussing how they&#x27;ve quantified the leak here.What would you like to see? The SAFE_CHAR logging allowed us to get data on the rate which is how I got the % of requests figure.

In case you&#x27;re wondering how this could be worse than Heartbleed:Yes, apparently the allocation patterns inside Cloudflare mean TLS keys aren&#x27;t exposed to this vulnerability.But Heartbleed happened at the TLS layer. To get secrets from Heartbleed, you had to make a particular TLS request that nobody normally makes.Cloudbleed is a bug in Cloudflare&#x27;s HTML parser, and the secrets it discloses are mixed in with, apparently, HTTP response data. The modern web is designed to cache HTTP responses aggressively, so whatever secrets Cloudflare revealed could be saved in random caches indefinitely.You really want to see Cloudflare spend more time discussing how they&#x27;ve quantified the leak here.

What a mess.On the plus side, all those booter services hiding behind the Cloudflare are probably being probed and classified&#x2F;identified&#x2F;disabled by competitors and probably FBI. That is good.

Far worse than this. Yes, browser caches, but also web crawlers (like google)&#x27;s caches. This means that anyone who requested certain public content could have instead received secret content from completely unrelated websites.

There are all kinds of places were things are cached, both on- and offline. Your data may end up in:* Browser caches.* Sites like wayback machine or search engines that make copies of webpages and save them.* Tools that store data downloaded from the web, e.g. RSS readers.* Caching proxies.* the list goes on and on.I think what tptacek wanted to say: It&#x27;s just so common that people download things from the web and store them without even thinking much about it. And all those places where this happens now potentially can contain sensitive data.

There are caches all over the Internet; Google and Microsoft run some of them, but so do virtually every Fortune 500 company, most universities, and governments all over the world.The best way to understand the bug is this: if a particular HTTP response happened to be generated in response to a request, the response would be intermingled with random memory contents from Cloudflare&#x27;s proxies. If that request&#x2F;response happened through someone else&#x27;s HTTP proxy --- for instance, because it was initiated by someone at a big company that routes all its traffic through a Bluecoat appliance --- then that appliance might still have that improperly disclosed memory saved.

As for the SHA-1 collision mentioned by jgrahamc[1] earlier today:How am I going to explain this to my wife?Actually a serious question. How do we communicate something like this to the general public?[1] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=13713826" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=13713826</a>

It&#x27;s reminiscent of the earlier days of the Squid cache.When it had bugs and devivered up cached files the typical symptom was that everyone in the company got unwanted porn.Because the biggest user (by far) of the &#x27;net was the person into porn and so 90% of the Squid cache was porn.

Many services on the internet keep a copy of a page they have loaded in the past. Google does this, for example. It lets them do things like search across websites quickly.Many of these caches are available online, to anyone who wants to look at them.This bug meant that any time a page was sent through Cloudflare, the requester might receive the page plus some sensitive personal information, or credentials that could be used to log in to a stranger&#x27;s account. Some of these credentials might let a bad actor pretend to be a service like Uber or Fitbit.This very sensitive information might end up saved in a public cache, where anyone could find it and use it to do harm.

Can someone tell me the implications of this in laymen terms?For instance what does it mean &quot;sprayed into caches&quot;? what cache? dns cache? browser cache? if the latter, does it mean you are safe if the person who owns that cache is an innocent non technical iser?

Wow, I just tried this, the first result with a google cache copy has a bunch of the kind of data described.  Although there was only one result with a cache.

&gt; The infosec team worked to identify URIs in search engine caches that had leaked memory and get them purged. With the help of Google, Yahoo, Bing and others, we found 770 unique URIs that had been cached and which contained leaked memory. Those 770 unique URIs covered 161 unique domains. The leaked memory has been purged with the help of the search engines.So I tried it too, and there&#x27;s still data cached there.Am I misunderstanding something - that above statement must be wrong, surely?They can&#x27;t have found everything even in the big search engines if it&#x27;s still showing up in Google&#x27;s cache, let alone the infinity other caches around the place.EDIT: If the cloudflare team sees I see leaked credentials for these domains:android-cdn-api.fitbit.comiphone-cdn-client.fitbit.comapi-v2launch.trakt.tv

<a href="https:&#x2F;&#x2F;webcache.googleusercontent.com&#x2F;search?q=cache:lw4K9G2F1WgJ:lightnetwork.ph&#x2F;ofw-family-day-december-1&#x2F;&amp;num=1&amp;hl=en&amp;gl=uk&amp;strip=0&amp;vwsrc=1" rel="nofollow">https:&#x2F;&#x2F;webcache.googleusercontent.com&#x2F;search?q=cache:lw4K9G...</a><pre><code> Internal Upstream Server Certificate
 ...
 &#x2F;C=US&#x2F;ST=California&#x2F;L=San Francisco&#x2F;O=Cloudflare Inc.&#x2F;OU=Cloudflare Services - nginx-cache&#x2F;CN=Internal Upstream Server Certificate
</code></pre>
That really doesn&#x27;t look good.

The first couple I looked at were requests to Uber and Fitbit...

Lol, Google just purged that search.EDIT: but there&#x27;s still plenty of fish: <a href="http:&#x2F;&#x2F;webcache.googleusercontent.com&#x2F;search?q=cache:lw4K9G2F1WgJ:lightnetwork.ph&#x2F;ofw-family-day-december-1&#x2F;&amp;num=1&amp;hl=en&amp;gl=uk&amp;strip=0&amp;vwsrc=1" rel="nofollow">http:&#x2F;&#x2F;webcache.googleusercontent.com&#x2F;search?q=cache:lw4K9G2...</a>This will take weeks to clean, and that&#x27;s just for Google.EDIT2: found other oauth tokens, lots of fitbit calls... And this just by searching for typical CF internal headers on Google and Bing. There is no way to know what else is out there. What a mess.

The &quot;well-known chat service&quot; mentioned by Tavis appears to be Discord, for the record.edit: Uber also seems to be affected.

more results from duckduckgo: <a href="https:&#x2F;&#x2F;duckduckgo.com&#x2F;?q=+%7B%22scheme%22%3A%22http%22%7D+CF-Host-Origin-IP&amp;t=h_&amp;ia=web" rel="nofollow">https:&#x2F;&#x2F;duckduckgo.com&#x2F;?q=+%7B%22scheme%22%3A%22http%22%7D+C...</a>

this is quite bad. i hope google can put some effort in clearing it&#x27;s cache too

I searched for&quot;CF-Host-Origin-IP:&quot; token.... uhm is that what I think I&#x27;m seeing???

It seems like the reasonable thing for Google to do is to clear their entire cache.  The whole thing.   This is the one thing that they could do to be certain that they aren&#x27;t caching any of this.

&gt;It is a snapshot of the page as it appeared on Feb 21, 2017 20:20:45 GMTSo the issue wasn&#x27;t fully fixed on Feb 19, or Google&#x27;s cache date isn&#x27;t accurate?

Time to find out where various &quot;booter&quot; sites are actually hiding.

It is far from over, too! Google Cache still has loads of sensitive information, a link away!Look at this, click on the downward arrow, &quot;Cached&quot;: <a href="https:&#x2F;&#x2F;www.google.com&#x2F;search?q=&quot;CF-Host-Origin-IP:&quot;+&quot;authorization:&quot;" rel="nofollow">https:&#x2F;&#x2F;www.google.com&#x2F;search?q=&quot;CF-Host-Origin-IP:&quot;+&quot;author...</a>(And then, in Google Cache, &quot;view source&quot;, search for &quot;authorization&quot;.)(Various combinations of HTTP headers to search for yield more results.)

Isn&#x27;t it worse than that? Even if you are not a CF user, if your apps make calls to a third party site protected by CF, you could be at risk (stolen credentials, API keys), and could be attacked using those now.

&gt; If anyone here is HIPAA-regulated or you have a customer who isCloudflare certainly does; I founded a health tech company, and Cloudflare was the recommended go-to for health tech startups who needed a CDN while serving PHI.And this is definitely a reportable breach. Technically any breach is supposed to be reported to HHS, but in reality, a lot of covered entities (e.g. insurers) fail to report smaller breaches (which, as a patient, should terrify you). The big ones, though, are really, really bad, and when reported, the consequences can be very serious and potentially even include serving time, depending on the circumstances.The reason I can be so confident that this is a reportable breach is that the definition of PHI is so broad that even revealing the existence of information between two known entities can be considered protected information. Anything more specific, like a phone number or DOB, or time of an appointment (even if you don&#x27;t know who the appointment corresponds to) - that&#x27;s always protected. And Cloudflare certainly has many of those.

If you are&#x2F;were using Cloudflare to cache PHI though their CDN without a BAA, you were likely in breach before this.Some have suggested that Cloudflare might not be a business associate because of an exception to the definition of business associate known as the &quot;conduit&quot; exception.Cloudflare is almost certainly not a conduit. HHS&#x27;s recent guidance on cloud computing takes a very narrow view[0]:&quot;The conduit exception applies where the only services provided to a covered entity or business associate customer are for transmission of ePHI that do not involve any storage of the information other than on a temporary basis incident to the transmission service.&quot;OCR hasn&#x27;t clarified what &quot;temporary&quot; means or whether a CDN would qualify, but again, almost certainly not. ISPs qualify, but your data just sits on the CDN indefinitely.p.s. Hi Patrick and Aditya![0] <a href="https:&#x2F;&#x2F;www.hhs.gov&#x2F;hipaa&#x2F;for-professionals&#x2F;special-topics&#x2F;cloud-computing&#x2F;index.html" rel="nofollow">https:&#x2F;&#x2F;www.hhs.gov&#x2F;hipaa&#x2F;for-professionals&#x2F;special-topics&#x2F;c...</a>

If anyone here is HIPAA-regulated or you have a customer who is, and you used Cloudflare during those dates, it is Big Red Button time. You&#x27;ve almost certainly got a reportable breach; depending on how tightly you&#x27;re able to scope it maybe it won&#x27;t be company-ending.

&gt; This is approximately as bad as it ever gets.*as bad as it has ever gotten so far.

It&#x27;s as easy as throwing a red banner on your website that explains the situation briefly and recommends that users change their passwords, if you take this more seriously you can force a password reset for all users. Depends on how sensitive the information that your users trust your site to hold is.

Email your customers, telling them to change their passwords, and link to some info about the leak. (in case they don&#x27;t visit your website and miss seeing the security alert banner)Advise them to change passwords for other services too, list sites possibly affected: <a href="https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare&#x2F;blob&#x2F;master&#x2F;README.md" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare&#x2F;blob&#x2F;master...</a>

&gt; A significant number of companies probably need to compose customer notifications;As a one-man company who has never done this before (and to the best of my knowledge never needed to): Any guides&#x2F;examples to writing a customer notification for security ups like this? Or just recommendations? Thanks.

&gt;Tavis found it by accident just looking through Google search results.Curious whether there could be some automated way of preventing such a widespread cache poisoning in the future. Some ML trained on valid pages from a given domain?Is it even possible to recover the original content of the documents or was the data randomly inserted into different parts?

Oh, my god.Read the whole event log.If you were behind Cloudflare and it was proxying sensitive data (the contents of HTTP POSTs, &amp;c), they&#x27;ve potentially been spraying it into caches all across the Internet; it was so bad that Tavis found it by accident just looking through Google search results.The crazy thing here is that the Project Zero people were joking last night about a disclosure that was going to keep everyone at work late today. And, this morning, Google announced the SHA-1 collision, which everyone (including the insiders who leaked that the SHA-1 collision was coming) thought was the big announcement.Nope. A SHA-1 collision, it turns out, is the minor security news of the day.This is approximately as bad as it ever gets. A significant number of companies probably need to compose customer notifications; it&#x27;s, at this point, very difficult to rule out unauthorized disclosure of anything that traversed Cloudflare.

If serious researchers are looking to get paid, I think bug bounties are the wrong approach entirely

I don&#x27;t disagree.But, Taviso is probably contractually prohibited from accepting money from CF as a Google employee. Many large companies have &#x27;outside activity&#x27; clauses and Google seems to be paying him already for that.However, it will affect others whom are fully freelance.

&gt; &quot;Why do we offer $5k for a small bug again? Look at CF, they don&#x27;t offer any money!&quot;Answer: &quot;Because if they had set up a bounty of $50k for security issues, they&#x27;d had thousands of researchers&#x2F;students&#x2F;white hats etc. watching the output of their servers.&quot;

Not only that, but the &quot;reward&quot; in the program is laughable and frankly insulting to any serious researcher considering the scope of CF. Bug bounty platforms are already becoming the fiverr of ITSEC (that&#x27;s not a good thing), CF just made an extra effort do diminish the value for researchers.Management: &quot;Why do we offer $5k for a small bug again? Look at CF, they don&#x27;t offer any money!&quot;

I never really got this argument. Is it not much better than the majority of companies that have no bug bounty and where the reporter needs to worry they will be met with legal threats instead of a t-shirt?

I got a t-shirt from cloudflare, and all i did was tell them &quot;please send me a t-shirt&quot; - they shipped it halfway across the world as well, for free! (it didn&#x27;t fit...)

Good to know the security of their users is worth a t-shirt

&gt;Cloudflare pointed out their bug bounty program, but I noticed it has a top-tier reward of a t-shirt.Considering the amount and sensitivity of the data they handle, I&#x27;m not sure a t-shirt is an appropriate top-tier reward.

Wow apparently they never fuzzed their input and looked at the output. A malformed html input should be about the easiest possible thing to try... yeouch...

Well, my day tomorrow is going to be busy. So&#x27;s my evening tonight, I guess.

Yes, but in general keys provide ongoing access; sensitive data itself is more limited in scope.  Keys, auth tokens, etc. would be what I&#x27;d focus on.

What can they even say? &quot;Change everything&quot; doesn&#x27;t really work. Any potentially secret data to or from a server could have been exposed.

&gt;I&#x27;m currently scrambling for remediation ideas. &quot;Change everything&quot; isn&#x27;t tractable.It&#x27;s not easy to deal with but it is the best remediation available to you, given the exceptionally broad scope and months-long period where data was apparently leaking (the cloudflare blog post lists 2016-09-22 as the first date when leaks were possible)

I&#x27;m scrambling to compile an list of impacted domains and some easy-to-understand instructions that I can send friends and family:<a href="https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare</a>

Right there with you. I&#x27;m currently scrambling for remediation ideas. &quot;Change everything&quot; isn&#x27;t tractable.

Neither this thread nor the Cloudflare blog post include concise steps for customers who were exposed.There&#x27;s an argument for changing secrets (user passwords, API keys, etc.) for potentially affected sites, plus of course investigating logs for any anomalous activity. It would be nice if there were a guide for affected users, maybe a supplemental blog post.(and yet again: thank you Google for Project Zero!)

Yup, this made it crystal clear, years ago, that Cloudflare&#x27;s business incentives were and are at odds with a secure web.

Step &quot;What does secure mean anyway&quot;) SSL terminate even sites that are not sending data to Cloudflare securely

So rather than demand fixes for the fundamental issues that enable ddos attacks (preventing IP spoofing, allowing infected computers to remain connected, etc), we just continue down this path of massive centralization of services into a few big players that can afford the arms race against bonnets.Using services like Cloudflare as a &#x27;fix&#x27; is wrecking the decentralized principles of the Internet. At that point we might as well just write all apps as Facebook widgets.

&gt;I&#x27;m not ready to throw out the baby with the bathwater because of one issue.Extreme centralization of the Internet is not a &quot;baby&quot;, except maybe in the sense of a cuckoo&#x27;s egg.But I&#x27;m willing to bet the mentality of this comment is highly representative of many web developers and service providers. They will not seek to fix anything, because they don&#x27;t see this state of things as a problem in the first place.

How about... stop CLOUD THIS and CLOUD THAT.Cloud means extreme centralization.It means giving your data to a third party you don&#x27;t control.Why?Why does our networked software have to assume a centralized topology?In the days when developed countries had dialup, protocols (IRC, Email, etc.) were all decentralized. Today, all the famous developers live with fancy broadband internet connections and forgot what it&#x27;s like to have to think about netsplits.The result... all the software is either &quot;online&quot; or broken.There shouldn&#x27;t be an &quot;online&quot; or &quot;offline&quot;. There should be &quot;do I have access to server X currently?&quot;Why do we need Google Docs to collaborate on a document if we are all in the same classroom?Why do we need centralized facebook server farms whose engineers post on highscalability how they enable us all to post petabytes of photos and comment to our friends?Why do we need centralized sites to comment at all? Each thread is local to its parent.Why does India need internet.org from facebook?If communities could have a network that survives without an uplink to the outside world then DDOS from the global internet would just cut off that network&#x27;s hosting of documents to outsiders. They&#x27;d still be able to do EVERYTHING locally - plan dinners, book a local appointment, send an email etc. and even post things out to the greater internet.This is a future I want to see.We already have mesh networks. We need more web based software to run these things.That&#x27;s what we are building at qbix.com btw.

&gt;edit: why the revulsionI&#x27;d guess it&#x27;s because of the crude and reductive way you describe the service cloudflare provides. I don&#x27;t know what type of programming you do, but many small services don&#x27;t have the infrastructure to mitigate the kind of attacks cloudflare deals with and they wouldn&#x27;t be around without services like this.I don&#x27;t like the internet becoming centralized into a few small places that mitigate DDOS attacks like this, but I like the alternative (being held ransom by anyone with access to a botnet) even less.I&#x27;m going to take a more even handed approach than what you&#x27;re suggesting. Any time you work with a service like this you risk these kinds of things - it&#x27;s part of the implicit cost&#x2F;benefit analysis humans do every day. I&#x27;m not ready to throw out the baby with the bathwater because of one issue. I&#x27;m not sure what alternative you&#x27;re suggesting (I didn&#x27;t see any suggestions, just a lot of ranting, which might also contribute to the &#x27;revulsion&#x27;) but it doesn&#x27;t sound any better than what we have.

Long been rumoured in the more paranoid corners of the web they are intelligence front&#x2F;partners.

Covertly get into a company that terminates ssl for half the internet, and... spill your precious secrets everywhere, instead of siphoning them off silently?

&quot;Step 0) Obtain black funding from NSA budget to start and &quot;VC invest&quot; in a global CDN company...&quot;I once came up with that exact concept for a nation-state subversion. It would even pay for itself over time. I kept thinking back to it seeing the rise of the CDN&#x27;s and the security approaches that trust them.

Not NSA, but the CIA funds and operates In-Q-Tel[1]. They&#x27;ve funded companies like Palantir and Keyhole (which became Google Earth).[1] <a href="https:&#x2F;&#x2F;www.crunchbase.com&#x2F;organization&#x2F;in-q-tel" rel="nofollow">https:&#x2F;&#x2F;www.crunchbase.com&#x2F;organization&#x2F;in-q-tel</a>

Am I misunderstanding that this would be useful for parallel construction, but that the public failure actually subverts the usefulness of Cloudflare as a MTIM partnering with someone?

Step 0) Obtain black funding from NSA budget to start and &quot;VC invest&quot; in a global CDN company...(Now I&#x27;m trawling Crunchbase to see if I can work out which investors are NSA front companies, then I&#x27;m gonna look to see what _else_ them and their partners have invested in...)

&gt;with a memory safe language and prevented all issues like that from ever happening.drastically reduced, but not quite ever.
For instance, use a GC language, especially in this domain, you might do some data pooling to reduce GC overhead. Maybe you forget to clear data in the pool. Same kind of error can result.But yes, I feel like security sensitive stuff like this shouldn&#x27;t be done in C &#x2F; C++ any more.

&quot;This bug would have been caught with valgrind, and by the sounds of it, using nothing more complex than feeding their httpd a random sampling of live inputs for an hour or two&quot;Or prevented using abstraction that do bounds checking. Or even just used ragel with a memory safe language and prevented all issues like that from ever happening. Probably would have been less work even with the reimplementation of an http proxy from scratch.

To be fair to CloudFlare, Google had a heap issue a few years back (maybe like 7 now) where internal flags and copies of argv (which Google use heavily for config) were clearly present in output from their HTTP frontends, including references to Borg before Borg was ever documented publicly.Over in App Engine land, someone bypassed their JVM sandbox and managed to extract a copy of their JVM image, which included much of their revered base system statically linked into something like a 500mb binary.Sorry, I&#x27;d have to go digging to find references to either of these incidents. At least in either case customer data wasn&#x27;t leaking, but suffice to say it&#x27;s a little bit of the pot calling the kettle blackAnd finally let&#x27;s not forget the China incident, which rumour has it, resulted in a system compromise at Google right to the heart of their engineering organization. Of course they didn&#x27;t get roasted like Yahoo recently did over their password leak

CloudFlare is neither the first nor the biggest CDN. I can&#x27;t recall Akamai having a hole this big. They&#x27;re either more secure or better at keeping things quiet.

You&#x27;re absolutely right. Cloudflare is a &quot;global active adversary&quot;[1] and has done irreparable harm to the internet at large. This is just a small taste of what&#x27;s surely to come from CloudFlare&#x27;s massive influence. They&#x27;ve shown that they cannot be trusted with everyone&#x27;s data.[1] <a href="https:&#x2F;&#x2F;trac.torproject.org&#x2F;projects&#x2F;tor&#x2F;ticket&#x2F;18361" rel="nofollow">https:&#x2F;&#x2F;trac.torproject.org&#x2F;projects&#x2F;tor&#x2F;ticket&#x2F;18361</a>

It makes sense that they treat Tor like a probable adversary, but the cost analysis seems really flawed.Sure, the proportion of requests passing through Tor are more likely to be malicious, but given the bandwidth constraints the adversary seems limited.The costs aren&#x27;t only the lost business from people like you, but people who should use Tor giving in. There&#x27;s some wisdom to people even researching something as mundane as what their dog ingested using anonymized services, much less other medical questions.

I&#x27;m running a side-project on Cloudflare and it&#x27;s accessible through Tor without problems. I suspect this comes down to the settings a site owner sets up in their Cloudflare interface. It would stand to reason if for example you applied the highest security setting across the board, Tor and VPN users would get presented with a captcha.

They also actively deter Tor use. I&#x27;ve cancelled subscriptions with Cloudflare-hosted sites because they make securely and anonymously browsing their sites a pain.

Step 1) MITM the entire Internet, undermining its SSL infrastructure, build a business around itStep 2) leak cleartext from said MITM&#x27;d connections to the entire InternetI recently noted that in some ways Cloudflare are probably the only entity to have ever managed to cause more damage to popular cryptography since the 2008 Debian OpenSSL bug (thanks to their &quot;flexible&quot; &quot;&quot;SSL&quot;&quot; &quot;&quot;&quot;feature&quot;&quot;&quot;), but now I&#x27;m certain of it.&quot;Trust us&quot; doesn&#x27;t fly any more, this simply isn&#x27;t good enough. Sorry, you lost my vote. Not even onceedit: why the revulsion? This bug would have been caught with valgrind, and by the sounds of it, using nothing more complex than feeding their httpd a random sampling of live inputs for an hour or two

Oh that happened to my site as well. Was fun trying to figure out what was going on in the middle of the day.Nothing on their status page about it though :|

Our CNAME pointing to github pages was down on Cloudflare today with a 1014 error.  I&#x27;m guessing they broke some other stuff while scrambling to fix this privacy issue?  Not a good day for them.

Sounds bad to me...&quot;We&#x27;ve discovered (and purged) cached pages that contain private messages from well-known services, PII from major sites that use cloudflare, and even plaintext API requests from a popular password manager that were sent over https (!!).&quot;The trouble is you have no way to know if someone discovered this earlier, and harvested info for a long time.Or, how much harvested info from your site might be in a Google cache for someone else&#x27;s site.

Read Tavis&#x27; comments. He disagrees with you regarding the severity. This is a big f--king deal!

I don&#x27;t think he had to, but he got an answer in minutes. I don&#x27;t think that&#x27;s the part to be worried about.As for what you should do: it sounds like the impact is relatively low. I&#x27;d personally change easily-changed secrets which go over the session, and potentially externally facing customer passwords (yes in enterprise, maybe not in consumer).(I don&#x27;t have any insider info on this breach, though, but I read both posts and know how the system works.)

How does such a simple bug not get picked by auto tests, ci or end to end tests? I am baffled. Since we are behind cloudflare, I am not sure what I should tell my manager now. I lack the technical know how to parse that extremely technical article. Are we supposed to just assume all our traffic that passed via cloudflare is possibly compromised?It&#x27;s also a bit sad that travis has to contact cloudflare by twitter. Seriousy?Edit: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;taviso&#x2F;status&#x2F;832744397800214528" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;taviso&#x2F;status&#x2F;832744397800214528</a> is the tweet in question

&gt; One of the advantages of being a service is that bugs can go from reported to fixed in minutes to hours instead of months. The industry standard time allowed to deploy a fix for a bug like this is usually three months; we were completely finished globally in under 7 hours with an initial mitigation in 47 minutes.Great, that makes me feel so much better! I&#x27;m sorry, don&#x27;t try to put a cherry on the top when you&#x27;ve just leaked PII and encrypted communications.Additionally, most vendors in the industry aren&#x27;t deployed in front of quite as much traffic as CloudFlare is. It&#x27;s a miracle that ProjectZero managed to find the issue.

The quoted part that specifically mentions logged urls containing query strings has nothing to do with logging?

During debugging, that seems fine. The previous sentence makes it clear that they added specific logging to track down the problem. I&#x27;d rather have a process that allows engineers debugging memory corruption to see the data that&#x27;s in the process they&#x27;re debugging, than a process that prohibits them from seeing it.

&gt; Many of the logged urls contained query strings from https requests that I don&#x27;t think they intended to share.I guess this confirms a few things.- The complete query strings are logged,- They don&#x27;t appear to be too concerned with who accesses the logs internally or have a process that limits the access, and- They&#x27;re willing to send those logs out to a random person.

Ragel shares part of the blame. Why did it use a strict equality check when it could have trivially done a &gt;=?

I think this bug is kind of an indictment of Ragel. It has some great ideas, but since the generated code is so low level - and allows arbitrary blocks of code to be executed in the guts of the parser, bugs like these can result in this horrible memory issues - particularly since the generated code is often used to parse untrusted user input.

I&#x27;m fairly sure that it&#x27;s Discord.

Connecting some dots, I&#x27;m wondering if the &quot;well-known chat service&quot; is Slack:<a href="http:&#x2F;&#x2F;www.computing.co.uk&#x2F;ctg&#x2F;news&#x2F;2462266&#x2F;whatsapp-reddit-and-slack-knocked-offline-because-of-cloudflare-problems" rel="nofollow">http:&#x2F;&#x2F;www.computing.co.uk&#x2F;ctg&#x2F;news&#x2F;2462266&#x2F;whatsapp-reddit-...</a>

Some important parts:<pre><code> The examples we&#x27;re finding are so bad, I cancelled some
 weekend plans to go into the office on Sunday to help
 build some tools to cleanup. I&#x27;ve informed cloudflare
 what I&#x27;m working on. I&#x27;m finding private messages from
 major dating sites, full messages from a well-known
 chat service, online password manager data, frames from
 adult video sites, hotel bookings. We&#x27;re talking full
 https requests, client IP addresses, full responses,
 cookies, passwords, keys, data, everything.

 Cloudflare pointed out their bug bounty program, but I
 noticed it has a top-tier reward of a t-shirt.

 Cloudflare did finally send me a draft. It contains an 
 excellent postmortem, but severely downplays the risk
 to customers.</code></pre>

Do they have a t-shirt awards in their bug bounty programs?

Never been so relieved my company uses a different CDN...

Yep, even Tavis said it &quot;severely downplays the risk&quot;.

Is that because, even though a very small number of pages (they claim) triggered the bug, any adjacent traffic in memory could be disclosed?That traffic could be basically anything sent through Cloudflare, it would seem.

The implied comparison to Heartbleed problem is that everyone&#x27;s old encrypted traffic was suddenly in the open, key change didn&#x27;t help.(except for the enlightened few who used PFS before Heartbleed)

&quot;Don&#x27;t worry, the keys weren&#x27;t compromised.&quot;I know how to replace my TLS keys. I have no idea how to replace everything else.It&#x27;s like people who think losing my credit card number is the worst thing. No, it can be a hassle, but once I replace it I&#x27;m okay. It&#x27;s everything else.

If the bug exposes random uninitialized memory, can&#x27;t it affect a lot more sites?And if it truly is only ~3000 sites, where&#x27;s the list?

Only three thousand something sites [were potentially serving private data from all 7 million customer domains we host]

Cloudflare&#x27;s announcement, as it is currently worded, deserves the understatement-of-the-centry award.

My password manager has &gt; 500 entries. Changing all the passwords....isn&#x27;t going to happen any time soon.If it only took 60 seconds per site, it would still take eight hours to change them all.Might change a few key passwords, though. Couldn&#x27;t hurt. I only have a couple of bank&#x2F;financial passwords at this point. And my various hosting service access passwords.Anything else is not worth the hassle -- and mostly would have 2FA anyway.

How do you check if a website uses cloudflare ? Any scripts that do that ?

So it&#x27;s fixed, then? (I haven&#x27;t read the article yet.)

Note that for sites like HN, changing your password doesn&#x27;t expire other sessions. You have to go find every browser with an HN cookie and logout.(Where I mean some other sites that are not at all HN, but might plausibly exist.)

&gt; My second thought after relief was the realization that even as a consumer I&#x27;m affected by this, my password manager has &gt; 100 entries what percentage of them are using CloudFlare? Should I change all my passwords?Yes. Right now. Don&#x27;t wait for the vendor to notify you.&gt; What an epic mess. This is the problem with centralization, the system is broken.Yep.

You don&#x27;t have to use scrapers, just use copies of the TLD zone files looking for cloudflare nameservers.

As an aside, I found this really interesting:ashleymadison.comashleyrnadison.comI find it really interesting that they registered that particular misspelling and they both point to the same servers. I can see doing this for some obvious domains like gogle.com, but the distinction there is simply that r+n looks like m.Probably a really obvious answer here, but my guess is that they are trying to help people throw off the scent of someone browsing a history.

We&#x27;re compiling a list of domains using several scrapers and updating it here: <a href="https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare</a>You can start by cross referencing your password manager with this list, and working your way out from there.

&gt; You&#x27;ve got no idea what has been leakedIf your site is served through Cloudflare, assume it&#x27;s all out there because it might be. Standard Big Red Button(tm) procedure.I don&#x27;t run any particularly impressive sites but I&#x27;ll be resetting passwords today. Also cycling things I use behind Cloudflare like DigitalOcean passwords&#x2F;API keys.It&#x27;s supposed to be read-only Friday, Cloudflare :(

The damage is still yours even if its not your fault

If someone runs a red light, broadsides you while you&#x27;re in the intersection, and leaves you paralyzed... it is their fault both morally and legally... but it still sucks to be you since you bear the consequences regardless of fault.While this event is orders of magnitude less severe than my example, depending on the service that could be compromised there can be sufficient repercussions that you could not be made whole or avoid on-going inconvenience through the legal system or other acts of the genuinely responsible party.I absolutely get and sympathize with where you&#x27;re coming from... but you may want to check a few of your more important accounts none-the-less :-)

I won&#x27;t take the initiative of changing passwords, and I will only be doing it for services that ask me to do it.In my opinion, if my accounts get compromised because the provider uses Cloudflare and leaks my data all over, it&#x27;s their fault, not mine... It&#x27;s not my job to guess which services are using Cloudflare, which ones were affected... and further, if my account gets compromised, others presumably will.(PS: Of course you may need to change passwords if you reuse passwords from one service to the other, but obviously you shouldn&#x27;t be doing that in the first place.)

Wrong.  You need to re-read the disclosure.

They said https never broke, so if you were doing things right way you should not be affected at all. Do not overreact.

My first thought was relief, thank god I&#x27;m not using Cloudflare.Where would you even start to address this? Everything you&#x27;ve been serving is potentially compromised, API keys, sessions, personal information, user passwords, the works.You&#x27;ve got no idea what has been leaked. Should you reset all your user passwords, cycle all or your keys, notify all your customers that there data may have been stolen?My second thought after relief was the realization that even as a consumer I&#x27;m affected by this, my password manager has &gt; 100 entries what percentage of them are using CloudFlare? Should I change all my passwords?What an epic mess. This is the problem with centralization, the system is broken.

Compiling a list here: <a href="https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare</a>It will be updated as we find more.

Is there a list of sites potentially affected?I&#x27;m assuming I need to change my passwords on a significant number of sites. So far none of them have alerted me to a potential breach. Would love to have a head start.

taviso called out 1Password in the initial tweet, although that still leaves room for other password managers too: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;taviso&#x2F;status&#x2F;834900838837411840" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;taviso&#x2F;status&#x2F;834900838837411840</a>

According to Tavis Ormandy&#x27;s twitter post it was 1Password, <a href="https:&#x2F;&#x2F;twitter.com&#x2F;taviso&#x2F;status&#x2F;834900838837411840" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;taviso&#x2F;status&#x2F;834900838837411840</a>

Tavis specifically named 1Password, was that wrong?<a href="https:&#x2F;&#x2F;twitter.com&#x2F;taviso&#x2F;status&#x2F;834900838837411840" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;taviso&#x2F;status&#x2F;834900838837411840</a>

I just checked them too, but they may have just switched over.

I am confused - why is 1Password is using anything but iCloud or Dropbox? Those are the only options I see (and &quot;Folder&quot; which is presumably just local).

Thankfully it looks like it&#x27;s not 1Password, who seem to use AWS CloudFront.

It looks like it was 1Password who have blogged their take on this Cloudflare vulnerability here - <a href="https:&#x2F;&#x2F;blog.agilebits.com&#x2F;2017&#x2F;02&#x2F;23&#x2F;three-layers-of-encryption-keeps-you-safe-when-ssltls-fails&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.agilebits.com&#x2F;2017&#x2F;02&#x2F;23&#x2F;three-layers-of-encryp...</a>

Anyone know which password manager uses Cloudflare?  Just trying to figure out if I&#x27;m affected.

Just looking at that site... what&#x27;s so bad about Wikipedia?  There&#x27;s a lot to criticize about Wikipedia, but I&#x27;ve never heard of them violating someone&#x27;s privacy.

Related: <a href="http:&#x2F;&#x2F;crimeflare.com&#x2F;" rel="nofollow">http:&#x2F;&#x2F;crimeflare.com&#x2F;</a>

I can&#x27;t change my Uber password - first, the only way to do so is via the &#x27;Forgot your password&#x27; dialogue, and second, that now produces a 500 error from NGINX.Lots of services are going to crumple under the weight of frantic password-reset requests.

This is probably a good moment to recall the article I published a while ago about how CloudFlare is actively putting the web at risk: <a href="http:&#x2F;&#x2F;cryto.net&#x2F;~joepie91&#x2F;blog&#x2F;2016&#x2F;07&#x2F;14&#x2F;cloudflare-we-have-a-problem&#x2F;" rel="nofollow">http:&#x2F;&#x2F;cryto.net&#x2F;~joepie91&#x2F;blog&#x2F;2016&#x2F;07&#x2F;14&#x2F;cloudflare-we-hav...</a>This is precisely why. The only thing that surprises me about this, is that it was an accidental disclosure rather than a breach. Other than that, this was completely to be expected.EDIT: Also, this can&#x27;t be repeated enough: EVERYBODY IS AFFECTED. Change your passwords, everywhere, right now. Don&#x27;t wait for vendors to notify you.Anything could have irrevocably leaked, and you have no way of knowing for sure, so assume the worst.

Popular companies that use CloudFlare can be found here: <a href="https:&#x2F;&#x2F;stackshare.io&#x2F;cloudflare&#x2F;in-stacks" rel="nofollow">https:&#x2F;&#x2F;stackshare.io&#x2F;cloudflare&#x2F;in-stacks</a>

I know I could find out by checking DNS, etc., but I&#x27;d rather not have to do that for every web site I use... and I&#x27;m guessing Cloudflare doesn&#x27;t publish a list of every domain name that they serve.

You don&#x27;t even have to write a script, all that info is already out there.

Looks cool, you have a lot of duplicates, though.<pre><code> $ cat sorted_unique_cf.txt | wc -l
 7385121

 $ cat sorted_unique_cf.txt | uniq |wc -l
 4287625</code></pre>

Yup, running it now. Results are being posted as fast as I can here:<a href="https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare</a>

Not as epic, but I wrote a small node app that checks a list of URLs (made it to check my 1password sites, but it can be used on stuff in general). Not very sophisticated, just checks DNS and setCookie strings...<a href="https:&#x2F;&#x2F;github.com&#x2F;weltan&#x2F;cloudbleed-1password" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;weltan&#x2F;cloudbleed-1password</a>

Anyone wrote a script yet that checks the top 1M (or so) web sites to find out which use Cloudflare?  It would help with knowing what secrets I need to change (as an end user -- I&#x27;m not a Cloudflare customer, thank $deity).

It appears that it could be from any site that passed through that server, not just those with these features enabled.That&#x27;s just how I read it.

It only triggered the bug on sites that were using those features, but any other CF site was vulnerable to getting dumped out.

&gt; only sites with one or more of those features enabled leaked dataNo. From what he says, enabling that feature on a CF proxy basically triggered the bug on any site that happened to go through that proxy, regardless of whether it used the feature or not.

I&#x27;m not 100% clear: Only three features were affected, and only sites with one or more of those features enabled leaked data into their pages.But was the leaked data similarly limited to only the sites with the features enabled? Or could it have come from any request - even an entirely unrelated site?

Headers (among other sensitive stuff) were being leaked inside document bodies.

requesting a page with a specific combination of broken tags, when done through cloudflare, will cause neighboring memory to be dumped into the response. op suspects this is due to a bounds checking bug on a read or copy. one can imagine this can be potentially kilobytes of data in one go.since anyone can put a broken page behind cloudflare, all you need to do is request your own broken page through cloudflare, and start collecting the random &quot;secure&quot; data that comes back.

Everything is leaked, headers are common but full plaintext is leaked in some instances.

Cloudflare handles SSL for a lot of sites. It decrypts everything and passes it along.For certain other sites, with malformed html, there is a bug that caused it to grab random data (headers and body) from memory and include it in the body of the response HTML. (Some html rewriting product that cloudflare offered was broken and it ran on the same servers.)This stuff got sent to peoples browsers and also to web indexers like Google or Bing.Google lets you search for stuff and will also show you the original page that it scraped, making it easy to find this data.Edit: Also you may be seeing more headers in examples because headers are easier to search for.

HTTP Headers were being including the http response bodies of other random websites. Those websites were being crawled and cached.

I don&#x27;t get it. How is this info leaked? From the blog posts, it seems that &quot;only&quot; the HTTP Headers are being leaked and somehow being crawled by Google? But since when does Google store HTTP request info? Can someone explain?

Did Lastpass use Cloudflare? That would be a disaster.

Just deleted my LastPass account - have been converted to KeePass for over a month.

TL;DR for the lazy ones:&gt; The examples we&#x27;re finding are so bad, I cancelled some weekend plans to go into the office on Sunday to help build some tools to cleanup. I&#x27;ve informed cloudflare what I&#x27;m working on. I&#x27;m finding private messages from major dating sites, full messages from a well-known chat service, online password manager data, frames from adult video sites, hotel bookings. We&#x27;re talking full https requests, client IP addresses, full responses, cookies, passwords, keys, data, everything.This is huge.I mean, seriously, this is REALLY HUGE.

How do i get the key information to HN without passing through cloudflare so that someone could sniff it?

Everyone: HN should implement U2F or TOTP so we don&#x27;t need to only rely on passwords :(

See <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=13719437" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=13719437</a>

Can someone provide a lay-person&#x27;s explanation of the issue and its implications?

Salient question at this point: Did Cloudflare have any systems in place that would allow themselves to identify queries that were abusing this defect?

Only the paranoid survive, Andy Groves famously said.

Every piece of dependency in your stack is a vulnerability vector. I feel like this is the only sane assumption to make these days. Yesterday I was thinking of doing some stuff with cloudflare and today I&#x27;m reading this report.The modern web requires a paranoid attitude.

Where is a reliable list of CF-protected websites so I may identify which ones I have interacted with?

Some features had a bug which lead to uninitialized memory (AKA previous memory contents) in the output of a malformed HTML page was requested.As one such server handles many sites, everything that the server handled before that request may be compromised. This includes all HTTP-GET&#x2F;POST data (credentials, direct messages to other users, ...), Headers (API tokens, Login-Cookies) and contents.So, you have to assume that everything you did on a CF &quot;protected&quot; website in the last months (especially between 2017-02-13 and 2017-02-17) is potentially compromised.

I know what Cloudflare is but i don&#x27;t quite understand the underlying issue.Can someone explain in simpler terms what happened here and how it a) affects sites using Cloudflare and b) Users accessing sites with Cloudflare?

Cloudflare blog post related to this incident: <a href="https:&#x2F;&#x2F;blog.cloudflare.com&#x2F;incident-report-on-memory-leak-caused-by-cloudflare-parser-bug&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.cloudflare.com&#x2F;incident-report-on-memory-leak-c...</a>

I used Censys to search for the IPv4 addresses of servers serving matching TLS certificates: <a href="https:&#x2F;&#x2F;censys.io&#x2F;ipv4?q=443.https.tls.certificate.parsed.names%3A+*.ycombinator.com" rel="nofollow">https:&#x2F;&#x2F;censys.io&#x2F;ipv4?q=443.https.tls.certificate.parsed.na...</a>

(not op) Well as it happens, Cloudflare has had this massive data leak you might have heard of...

Hacker News uses Cloudflare: <a href="http:&#x2F;&#x2F;bgp.he.net&#x2F;dns&#x2F;news.ycombinator.com#_ipinfo" rel="nofollow">http:&#x2F;&#x2F;bgp.he.net&#x2F;dns&#x2F;news.ycombinator.com#_ipinfo</a>Add the following to your hosts file to bypass Cloudflare and access HN directly:<pre><code> 50.22.90.248 news.ycombinator.com</code></pre>

So you think businesses have no responsibility to police themselves and their users? If a shop was caught knowingly facilitating say welfare fraud they&#x27;d get fined up the wazoo but for some reason being ~digital~ makes it OK?

The thesis of that blog seems to be that Cloudflare should be censoring content and deciding who gets to have websites on the internet.

This might be the time to point out the CloudFlare watch blog: <a href="http:&#x2F;&#x2F;crimeflare.com&#x2F;" rel="nofollow">http:&#x2F;&#x2F;crimeflare.com&#x2F;</a>

it doesn&#x27;t seem to matter since _ANY_ service that CF gave data to could possibly have cached it and thus have a random site&#x27;s secrets.

While theoretically many non-cloudflare sites can be affected, I&#x27;m compiling a list of known cloudflare-using domains here: <a href="https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pirate&#x2F;sites-using-cloudflare</a>It can serve as a starting point, just Ctrl+F for domains you use.

So how does one find or generate a list of companies using CloudFlare to figure out how you&#x27;re affected - kinda like HaveIBeenPwned.com?

I was able to get a few hits from a quick google search that are still in google&#x27;s webcache.

Seems like we need to reset&#x2F;rotate everything. So yes, I believe so.

So time to reset password and logout of all mobile apps to get new authorization tokens?

It was a bug in C code (automatically generated by the Ragel state-machine compiler), some details are in CFs blog post: <a href="https:&#x2F;&#x2F;blog.cloudflare.com&#x2F;incident-report-on-memory-leak-caused-by-cloudflare-parser-bug&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.cloudflare.com&#x2F;incident-report-on-memory-leak-c...</a>

Their old Ragel-based parser was affected. According to their post mortem both the old parser and the new cf-html one are compiled as nginx modules so I&#x27;d venture a guess that this is probably C&#x2F;C++ code since afaik you can&#x27;t extend nginx through modules written in Go.&gt; It turned out that the underlying bug that caused the memory leak had been present in our Ragel-based parser for many years but no memory was leaked because of the way the internal NGINX buffers were used. Introducing cf-html subtly changed the buffering which enabled the leakage even though there were no problems in cf-html itself.<a href="https:&#x2F;&#x2F;blog.cloudflare.com&#x2F;incident-report-on-memory-leak-caused-by-cloudflare-parser-bug&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.cloudflare.com&#x2F;incident-report-on-memory-leak-c...</a>There&#x27;s a section in that blog post titled &quot;Root cause of the bug&quot; that goes in further detail.

Interesting. Cloudflare uses a lot of Go, which should hypothetically be memory safe. Was this system in Go? If so, I would be interested in seeing proof of concept code for a vulnerability like this.

Their clients, sure, especially they are HIPAA regulated (let&#x27;s pour one out for the poor sods) but CF only if everyone abandons them and many won&#x27;t. Gross negligence does not even exist online AFAIK and so criminally you can&#x27;t even start because there&#x27;s nothing to work with, perhaps negligence but that&#x27;s a slap on the wrist. A civil suit ... sure you can sue anyone in civil court for whatever but you need to prove damages here and that&#x27;ll be bloody hard.

Holy shit, this could be a company-ending event. For CloudFlare or any of its clients.

Then why are they talking about that 3000-ish number instead of the 7 million number?

Good at lying for sure <a href="https:&#x2F;&#x2F;twitter.com&#x2F;taviso&#x2F;status&#x2F;834918182640996353" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;taviso&#x2F;status&#x2F;834918182640996353</a>

CF&#x27;s infosec team is very, very good at their jobs.

People are going to lambast CF for downplaying the impact, and there could be merit in that.However, I really want to say I am absolutely impressed with both Project Zero AND Cloudflare on so many fronts, from clarity of communication, to collaboration, and rapid response. So many other organizations would have absolutely tanked when presented with this problem. Huge kudos for CF guys understanding the severity and aligning resources to make the fixes.In terms of P0 and Tavis though, holy crap. Where the heck would we be without these guys? Truly inspiring !

&gt; Cloudflare will protect any site that doesn&#x27;t host child pornDoesn&#x27;t that make it worse? They aren&#x27;t saying they don&#x27;t or won&#x27;t police the content they protect. They are obviously capable and willing to draw a line on ethical or legal grounds, if they have done so in that case. They have just chosen to draw that line on one side of porn but another side of DDoS services.Ultimately it is their decision to make, but I don&#x27;t think it&#x27;s unfair for people to question possible conflicts of interest in how that decision is made.

You don&#x27;t understand what freedom of speech actually means.

DDoS attacks are the ultimate form of censorship.

You are essentially arguing against freedom of speech.  Cloudflare will protect any site that doesn&#x27;t host child porn. Yes that includes things which you don&#x27;t like, but it also includes all the things you do.

That is how DDOS protection works, learning from data and scale to better defend future attacks. Every large network and security operator does this. What is your issue with that exactly?

Another implication: they could be using their access to these sites&#x27; traffic to prepare their own infrastructure for attacks before they happen.There&#x27;s nothing about their hosting of these sites that doesn&#x27;t reek.

Cloudflare sells protection from the internet attacks through its network. The same company and network facilitates the organisation of those same attacks, and helps keep them anonymous.That&#x27;s a high-tech protection racket.

In what way is this a protection racket? That&#x27;s sort of like complaining that mob-owned businesses enjoy the same police &amp; fire protection that all other businesses have.

It&#x27;s the same stance that antivirus developers have always had, more or less. As usual, the difference between blackhat and whitehat is very, very thin - if there is a difference at all.

Really? That search engine sells DDOS protection?

By the same logic, the search engine you used to find those sites is also a &quot;protection racket&quot;.

Cloudflare isn&#x27;t just a security hole in the middle of the internet, they&#x27;re a protection racket.If you wanted to pay to DDoS a site, search for &quot;booter&quot; and you&#x27;ll get a list of sites that will take another site off the internet for money with a flood of traffic.quezstresser.com
webstresser.co
topbooter.co
instabooter.com
booter.xyz
critical-boot.com
top10booters.com
betabooter.com
databooter.cometc. etc. - from the first 30 results I could find 2 booter sites that weren&#x27;t hosted by Cloudflare.But hey, pay Cloudflare and your site too can be safe from DDoS attacks...

Cloudflare Reverse Proxies Are Dumping Uninitialized Memory