&gt; Nested on the servers’ motherboards, the testers found a tiny microchip, not much bigger than a grain of rice, that wasn’t part of the boards’ original design.&gt; During the ensuing top-secret probe, which remains open more than three years later, investigators determined that the chips allowed the attackers to create a stealth doorway into any network that included the altered machines. Multiple people familiar with the matter say investigators found that the chips had been inserted at factories run by manufacturing subcontractors in China.

OpenBMC (Facebook, Google, Microsoft, Intel, IBM and others) is working on open-source baseboard management software, <a href="https:&#x2F;&#x2F;www.linuxfoundation.org&#x2F;blog&#x2F;2018&#x2F;03&#x2F;openbmc-project-community-comes-together-at-the-linux-foundation-to-define-open-source-implementation-of-bmc-firmware-stack&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.linuxfoundation.org&#x2F;blog&#x2F;2018&#x2F;03&#x2F;openbmc-project...</a>&gt; The organizations behind the new project each have already made substantial contributions to creating open source baseboard management controller (BMC) firmware. Now, working together, they will define the vision for a standard stack that can be used across systems and computing environments.LinuxBoot and Open Compute OSF are working on open-source server firmware that can be measured on every boot and validated against hardware root of trust keys controlled by the server owner instead of the server OEM, <a href="https:&#x2F;&#x2F;www.platformsecuritysummit.com&#x2F;2018&#x2F;speaker&#x2F;hudson&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.platformsecuritysummit.com&#x2F;2018&#x2F;speaker&#x2F;hudson&#x2F;</a>

It sometimes feels like certain hardware protocols were designed to be insecure. I remember reading about IPMI issues back in 2013:<a href="https:&#x2F;&#x2F;www.itworld.com&#x2F;article&#x2F;2708437&#x2F;security&#x2F;ipmi--the-most-dangerous-protocol-you-ve-never-heard-of.html" rel="nofollow">https:&#x2F;&#x2F;www.itworld.com&#x2F;article&#x2F;2708437&#x2F;security&#x2F;ipmi--the-m...</a>

Cannot it be physically disabled with a jumper or maybe by cutting some lines on the motherboard?

But without the IPMI kernel modules loaded, IPMI is harmless, right ?

If people are interested in digging into how IPMI works every SuperMicro board I&#x27;ve ever seen uses some variety of  ASPEED Baseboard Management Controller (BMC).  Facebook uses the same chip in their open hardware projects.

The problem with an IPMI BMC is that if you have malware that roots the OS from the BMC that in turn roots the BMC to reinstall itself from the OS, then you can never actually get rid of it. I actually opened a bug with a major clustering vendor in about 2012 because of this. Their response was a docfix.

&gt; The naive approach would be to not connect to the dedicated NIC that&#x27;s indicated on the back and in the instruction manual, but if you do this it masquerades onto the main NICcool, thanks for that info.&gt; just to write off the hardwaremaybe you could just standup the mgmt network but blackhole route it at each switch port. The mgmt NIC thinks it&#x27;s working properly but it can&#x27;t talk to anyone nor can anyone talk to it.at the expense of a dedicated switch.

SuperMicro hardware has very extensive IPMI integration into the mothebroard, which amongst other things can take over and inject frames into the network interface, emulate a VGA device, talk to the CPUs serial lines directly, flash firmware, control the state of a number of physical devices- and this is what it supports just from the web interface it presents by default with the password &quot;ADMIN:ADMIN&quot;. My money, based on experience attempting to harden their devices, is that any modification were injected into the IPMI hardware where most of this was already supported.This stuff ends up being extremely difficult to disable. The naive approach would be to not connect to the dedicated NIC that&#x27;s indicated on the back and in the instruction manual, but if you do this it masquerades onto the main NIC invisibly to the OS and DHCPs on its own to open up an administration port, web interface, and some assorted call homes. You have to explicitly tell it to use the non-connected port, change credentials, and modify it so that it is not accessible within operating system as well. Hopefully while the machine is offline to prevented any automated scanning finding it within your network.The number of times I&#x27;d end up nmaping our local networks and being able to remotely access production hardware with an interface that allowed me to reach this interface was maddening. The system is basically designed to be as insecure as possible by default, and allow for the maximum possible persistent threats with BIOS flashing, IPMI flashing, and other completely nu-authenticated avenues exposed. The course of action was always just to write off the hardware and bin it, because god knows what impact you could actually have using that interface.

It&#x27;s not particularly magical, there&#x27;s consumer chips around which are not a whole lot bigger (though obviously in a more standard package). You don&#x27;t get a lot of resources, but you don&#x27;t really need it if all the other frameworks are in place in other software. If this sort of thing is something you can buy on Mouser for a few cents, the espionage grade material is probably an order or magnitude more higher quality.<a href="https:&#x2F;&#x2F;www.microchip.com&#x2F;wwwproducts&#x2F;en&#x2F;ATtiny4" rel="nofollow">https:&#x2F;&#x2F;www.microchip.com&#x2F;wwwproducts&#x2F;en&#x2F;ATtiny4</a>For scale, this alone is about the size of a large SMD capacitor and would basically be lost in most designs today.

This sounds like speculation. I&#x27;m quite capable of coming up with my own unfounded speculation, but there is a real report out there with the actual details in that really needs to be made public, legally or otherwise. There ought to be a CVE about this. Where is it?

They didn&#x27;t do anything to the CPU, what they did is the modchipped the line from EEPROM and the board management controller.They probably found it out when they were repeatedly tried to reflash the BMC flash, and saw that checksums did not match.

Hm, but DMA messages get distributed over a parallel bus and this chip seems to employ a serial interface, so I would assume that it&#x27;s not directly connected to anything that requires high throughput (i.e. memory, disk and peripheral access).

<pre><code> 5. When a server was installed and 
 switched on, the microchip altered 
 the operating system’s core so it 
 could accept modifications. The chip 
 could also contact computers controlled 
 by the attackers in search of further 
 instructions and code.
</code></pre>
So, in typical vulnerability&#x2F;payload&#x2F;exploit fashion, the board&#x27;s bus is vulnerable by default, because the chip pierces all the usual lines of defense protecting against network and operator I&#x2F;O. It carries a payload intended to target very common features used everywhere commodity servers are used, one that likely listens for DMA traffic on the bus, and alters the signal stream, by escaping upon the occurrence of a magic sequence, and inserting its own signal, before resuming the authentic stream in flight.The payload could be pretty small, since the server boards are likely using OS packages that match the chipset. This limits the software to a small set of well known targets, Linux, Windows, Apple. Target their kernels, and you only have to snip out a small chunk of bytes, and splice your own pre-defined package in. Splice in a miniature runtime, that operates a turing complete set of operations, and open up a listener that waits for network access, and now, your payload can enable arbitrary code execution, irrespective of permissions.Now, to exploit, the payload needs to time the opportunity to splice itself onto the disk correctly. If certain well-known chunks of code will always exist in each given operating system, then with every disk access event one just needs to wait for the inevitable moment those magic system-specific bytes travel over the bus, in order to replace the known bytes with the poisoned modification. Events might target when the bytes are originally installed with the OS, or every time the OS reads those known bytes back into live memory, from any source.The total payload package could probably fit inside a couple of megabytes, pack on a few more for the &quot;listen &amp; splice&quot; part of the attack to round out the entire mass, and all we know how much data an SD card can fit into say... five grains of rice?

If this is indeed the case, I&#x27;m surprised someone didn&#x27;t catch something earlier when the device was calling &quot;home&quot; over their network. I&#x27;m wondering if China stole BlackRidge First Packet Authentication tech [1] to keep things dark. BlackRidge is... &quot;involved&quot; in IC and defense projects.[1]<a href="https:&#x2F;&#x2F;patents.google.com&#x2F;patent&#x2F;US8346951B2&#x2F;en" rel="nofollow">https:&#x2F;&#x2F;patents.google.com&#x2F;patent&#x2F;US8346951B2&#x2F;en</a>

The article says:But they were capable of doing two very important things: telling the device to communicate with one of several anonymous computers elsewhere on the internet that were loaded with more complex code; and preparing the device’s operating system to accept this new code. The illicit chips could do all this because they were connected to the baseboard management controller, a kind of superchip that administrators use to remotely log in to problematic servers, giving them access to the most sensitive code even on machines that have crashed or are turned off.To me, that makes it sound like they could download from a remote host and inject code and do literally anything.

Not the chip itself but there&#x27;s an investigation which I&#x27;ve already submitted:<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=18144519" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=18144519</a>

You&#x27;re not the only one worried about Lenovo.<a href="https:&#x2F;&#x2F;freebeacon.com&#x2F;national-security&#x2F;military-warns-chinese-computer-gear-poses-cyber-spy-threat&#x2F;" rel="nofollow">https:&#x2F;&#x2F;freebeacon.com&#x2F;national-security&#x2F;military-warns-chin...</a>

Is there an article that describes a bit more in detail what the chips actually did (or were capable of doing)? They only say &quot;the microchip altered the operating system’s core so it could accept modifications.&quot;, which I might interpret as circumventing signature checks to allow installing modified firmware on the systems? But how does the chip connect to the network and how does it receive commands?That said, it&#x27;s pretty scary that you can hide so much malicious functionality in such a small device, makes me wonder what might be hidden in my Lenovo. In any case it speaks highly of the auditing firm that they were able to locate this. I wonder if they performed an x-ray analysis of the board, as given the size of these chips it should be possible to embed such devices in one of the internal layers of the board as well, making them essentially invisible to optical inspection.

My point is that you should care about privacy, even if you&#x27;re using, for example, apple services. Thinking that nobody has the resources to hack a company that big isn&#x27;t an argument and has just been disproven.In other words, if you have plans to ever be a politician that china may not want in any position of power, don&#x27;t store your nudes on icloud&#x2F;dropbox&#x2F;google drive, or they may suddenly get leaked when you least expect it and ruin your career in favor of a more... Shall we say &quot;convenient&quot;? ...alternative.

I take a different view of this. Some 30 companies got this hardware, two are named as having the resources to find the offending hardware. By this logic you’re still safer using a company such as  Amazon.

I will certainly show this to everybody who tells me &quot;google won&#x27;t get hacked&quot; when privacy comes up :)Talking about things like this <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=18074097" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=18074097</a>

I stand corrected, thanks.Supermicro&#x27;s vendors they use in their Chinese manufacutring are the likely vectors for the implantation.

&gt; Supermicro could similarly be directed by Chinese national security interests to protect plausible deniability.Supermicro is an American corporation, headquartered in San Jose. They&#x27;re not directed by Chinese national security interests, they&#x27;ll do anything the US Government tells them to do when it comes to US national security.

The denials by Amazon, Apple, Supermicro and the Chinese Ministry of Foreign Affairs [1] are relatively pro forma, both directed by respective nation states involved in this matter. One of the reporters interviewed on Bloomberg noted Amazon and Apple could be directed by US national security interests to deny to protect the ongoing US investigation. Supermicro could similarly be directed by Chinese national security interests to protect plausible deniability.There was a sense of realpolitik by one UK guest commentator on Bloomberg, comments along the lines of &quot;hey, spying happens since time immemorial, put on some big boy pants, yes there is shock but not horror when the Snowden revelations came out, the US does it, too, etc.&quot;. I disagree with this sentiment, as while the attack was quite targeted, it puts into question a quite large supply chain network.Kudos to Bloomberg putting in the 12+ month investigative journalism to pull off this scoop. Yet another validation of the reasons I subscribe to listening&#x2F;watching them.[1] <a href="https:&#x2F;&#x2F;www.bloomberg.com&#x2F;news&#x2F;articles&#x2F;2018-10-04&#x2F;the-big-hack-amazon-apple-supermicro-and-beijing-respond" rel="nofollow">https:&#x2F;&#x2F;www.bloomberg.com&#x2F;news&#x2F;articles&#x2F;2018-10-04&#x2F;the-big-h...</a>

Which companies performed the security audits?

I’d expect that this type of parasite hardware attack isn’t unique. Securing the supply chain is even more important than ever.Apple’s response isn’t great IMHO.

Fascinating. A company I used to work for dismissed this sort of situation as a problem and ended up using SuperMicro boards in a security crucial product. Their hardware has always been notably very crappy, with the IPMI interface defaulting to world-open unsafe parameters, but I&#x27;d not expected it to be this cleverly hardware backdoored. It&#x27;s possible to neuter Intel ME, but that&#x27;s only a small comfort with these motherboards.

Good point. Definitely seems designed for mobile (only).

yep I have blurry vision coming back to HN for the comments...

I lasted about 3 seconds before I toggled reader view.

I found it perfect on my OLED mobile screen.OLED works better with darkness than with bright backgrounds.

&quot;People with astigmatism (approximately 50% of the population) find it harder to read white text on black than black text on white. Part of this has to do with light levels: with a bright display (white background) the iris closes a bit more, decreasing the effect of the &quot;deformed&quot; lens; with a dark display (black background) the iris opens to receive more light and the deformation of the lens creates a much fuzzier focus at the eye.&quot; - Jason Harrison – Post Doctoral Fellow, Imager Lab Manager – Sensory Perception and Interaction Research Group, University of British ColumbiaThis quote describes the situation well I think, why the opinions differ and why neither dominates UIs really.

This is because, contrary to what a lot of people say about dark themes (even though it&#x27;s mostly a meme at this point), dark letters on white background are better to read than white letters on black background.

Who else found the design of the page made the article difficult to read? I know darkness, spies and hacking go hand-in-hand but it&#x27;s a bit too much imho.

Right so all the stars need to align for it to go unnoticed - compromised server, firewall and other alerting&#x2F;monitoring tools.I would have thought one single unexpected packet in these high security environments would raise significant alarm bells and any anomaly would be found very quickly.

Who&#x27;s saying the firewall isn&#x27;t compromised too? ;)

I suppose the hardware on the navy ship is communicating with a server that is also compromised, and so on and so forth.

Firewalls in high security environments aren&#x27;t just port&#x2F;protocol based. You lock everything down - source ip&#x2F;port and destination ip&#x2F;port. You should know where it is coming from and where it is going to.Navy ships don&#x27;t upload via Dropbox.

Effectively nothing can be constrained by a whitelisting firewall if you have a sufficiently bored actor. You can smuggle data through a variety of benign looking protocols, things that wouldn&#x27;t matter in the least generally. Your average server contacts hundreds of different public NTP servers, binary repositories, domain name servers every day. If the keys to the kingdom are a 32 byte ECDSA private key, you&#x27;ve lost if you think you can protect this from reaching the outside world.A method that wouldn&#x27;t show up on any firewall in the world is simply to delay or drop certain SYN packets. Even if you only intended to transmit a bit at a time through this, any unauthenticated host on the internet could use this without raising any suspicion or even printing log lines in most environments. As soon are you&#x27;re making an assumption that you&#x27;re trying to prevent what&#x27;s inside from getting out things become substantially closer to impossible than anybody would like.

Presumably it contacts somewhere ubiquitous like AWS or Cloudflare.

Why aren&#x27;t these attacks constrained by normal corporate firewalls? How does a random server on a navy ship start contacting baddie.china.com without raising red flags?

It seems surreal to me that this could be real. And yet I&#x27;m not surprised.

Most likely it is using SPI <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Serial_Peripheral_Interface" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Serial_Peripheral_Interface</a>, that requires four pins and the two remaining ones are power and ground. SPI what is used to access EEPROMs and flash memory, so an attack that you can do is daisy chain such a device in the path to the EEPROM the board management controller uses as its firmware storage. Then you can very easily insert your own instructions and get the board management controller to execute whatever you want.

This sounds like something out of a movie. I would like some technical details how this is supposed to work with only a 6 lead chip.

&gt; Two of Elemental’s biggest early clients were the Mormon church, which used the technology to beam sermons to congregations around the world, and the adult film industry, which did not.

Yeah it seemed to imply a ‘tit for tat’ attitude to me

China is sourcing virtually nothing from US. If US is attacking the supply chain of China, it&#x27;s doing it on Chinese soil. Which brings us back to my interpretation

I Read that as &quot;The US is also attacking our hardware supply chains&quot;. That is, the statement concerned supply chain attacks in general, not this specific one.

<pre><code> The Chinese government didn’t directly address questions about manipulation of Supermicro servers, issuing a statement that read, in part, “Supply chain safety in cyberspace is an issue of common concern, and China is also a victim.” 

</code></pre>
Essentially China ils saying &quot;it was not me&quot;. Plausible

&gt; logic would point at the USBut then, since the logic points to the US it only stands to reason that is a false false flag! Why those wiley spies in Beijing really are clever. Discredit The USA and plant spy chips in important computers!

The US has a few things to gain from this story: Economically, because Chinese products are perceived as compromised. Politically: because the Chinese government is seen in the offense.The US has something to loose too: Being perceived as dependent on Chinese manufacturing and potentially compromised down to military hardware. (The first everybody knows, the second would be devastating for trust.)All in all it would be a weird angle for a false-flag attack.

&gt; ...but not a single time is it questioned who they planted it for...The article and some accompanying reporting on Bloomberg audio&#x2F;video segments says the attack seems targeted relatively specifically towards nearly 30 organizations (only US-based organizations were mentioned as targets, unknown if the list included organizations based in other nations). One known vector was through four subcontractors in China that built the boards for Supermicro&#x27;s main Shanghai factory, specifically by bribing and&#x2F;or coercing managers of those subcontractors&#x27; factories to go along with accepting the chip shipments and to make changes to the plant floor from the design to perform the chip insertions.Designing and building a chip like this and then mounting the logistical effort to performing the insertions costing some non-trivial funds, coupled with the known targets, (Amazon didn&#x27;t seem specifically targeted, Elemental a company they acquired was, who notably has US national security clients), form the circumstantial allegation that a PLA spy unit was behind the attack. You are correct that this doesn&#x27;t entirely rule out a false flag possibility, but until we get more details about this, we&#x27;re operating in the dark.A false flag is an interesting supposition, but how would the US benefit from successfully convincing the world of the false flag&#x27;s cover story?

The article specifically mentions the PLA as the bad guy. So that part was not left out. The whole story is hearsay so far. It&#x27;s plausible, but best read with a good dose of salt.If we believe the story up to the point where subcontractors are forced into planting the chips, we must accept that it&#x27;s easiest and least risk for Chinese government actors to force them into it. Mounting a false-flag attack at Chinese subcontractors would be exceedingly difficult for foreign agents. They&#x27;d likely blow their cover when trying to represent Chinese officials.If you want an alternate version, I suggest you start with the easiest: The whole thing didn&#x27;t happen. And Bloomberg is a victim of propaganda.

looks like no one noticed the word &quot;Troll&quot; in OP&#x27;s username

The only interesting thing about this is left out. Who planted it is clear (someone told to do so) but not a single time is it questioned who they planted it for. Smells like false flag to me.We think China does X Y and Z but we know the US does X Y Z and the rest of the alphabet. So unless something specific is leaked that shows who actually ordered this, logic would point at the US.

Probably much bigger. It&#x27;s naive to assume Supermicro was the only vendor compromised in this way. Similar implants probably exist in equipment from every major vendor of information-processing equipment. Routers, mobile network equipment, you name it.This is the leverage you give another nation-state when you let them control your supply chain.

This sounds strange but it&#x27;s commonplace to make parts like this. Some DCDC converter modules have silicon dies embedded into their PCBs to save physical footprint, most notably there&#x27;s a family of Texas Instruments ones which use this technique.<a href="https:&#x2F;&#x2F;www.electronicdesign.com&#x2F;sites&#x2F;electronicdesign.com&#x2F;files&#x2F;uploads&#x2F;2015&#x2F;12&#x2F;0716_TI_DC-DCmodules_F2.gif" rel="nofollow">https:&#x2F;&#x2F;www.electronicdesign.com&#x2F;sites&#x2F;electronicdesign.com&#x2F;...</a>

You&#x27;re right.One question though. Would a politically correct, by-the-book US president have had the balls to sanction China? Considering such sanctions could affect the US economy.

IC fabrication and packaged component assembly are very different things. China doesn&#x27;t have anything near this level of sophistication, at least not without any outside help.

&gt; I hope this corrects the mistaken believe that China can&#x27;t home grow sophisticated tech.There&#x27;s nothing particularly sophisticated about what they did, especially given what China has access to as one of the central hubs of tech manufacturing. There are two dozen nations (or more) that could do this from a strictly technical standpoint (few have the kind of required supply chain access to pull it off at scale in actuality). It&#x27;s the audacity that is primarily impressive. China is encouraging all of their richest trading partners to further isolate them when it comes to supply chain and advanced tech. They&#x27;re confirming for the 107th time what everyone already believes.

&gt;Amazon’s security team conducted its own investigation into AWS’s Beijing facilities and found altered motherboards there as well, including more sophisticated designs than they’d previously encountered.&gt;...the malicious chips were thin enough that they’d been embedded between the layers of fiberglass onto which the other components were attached&gt;...that generation of chips was smaller than a sharpened pencil tip, the person says.I hope this corrects the mistaken believe that China can&#x27;t home grow sophisticated tech.

My gut feel about this for many years has been outsourcing chip manufacturing to another country is a serious security risk. This was a hardware device, hard to detect as it was, how much code is in the chips we already expect on a motherboard?It seems flat out foolish for one country to own the world’s computer manufacturing.

I think the root of the problem is backwards compatibility and the fact that initially all these components were not designed with security in mind. So we&#x27;re adding more insecure components in spirit of &quot;move fast, break things&quot; than fixing the debt incurred.

&gt; It&#x27;s been a few years I&#x27;ve given up on the idea of privacy with technology.The problem with tech is that it&#x27;s modularity and dependency on other people&#x2F;tools&#x2F;hardware&#x2F;etc ultimately requires trust.&gt; in place for more than 20 years.Far longer than 20 years. The idea of trust, privacy and security has been discussed for a long time. The complexity of security on just on one specific technology ( compilers ) was discussed in the early 1980s by Ken Thompson.&quot;Reflections on Trusting Trust&quot;<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=13569275" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=13569275</a>Compilers are just one part of a complex ecosystem. Now imagine having to check ABI or the physical chips&#x2F;hardware themselves. Where you require sophisticated hardware.The motto of technology is &quot;In Trust We Trust&quot;.

This. I&#x27;ve even had an in-law say, &quot;My brother works for the Defense Intelligence Agency, and he uses smart devices in his home, so they must be safe!&quot;, with no consideration that tech may not be his specialty, or he doesn&#x27;t follow the daily IoT fiascos, or maybe he just thinks he won&#x27;t get hacked. Dunno. Meanwhile, my year-old thermostat still wants me to connect it to wi-fi, and that will never happen.

You&#x27;re definitely not a paranoid.However, IMO, the &quot;cost&quot; of ignorance such area is not that bad either. I will let them live their way to be honest.

The more I understand software the less I trust it (given the current state of engineering practices).  Meanwhile all my friends&#x2F;family are scrambling to install all the latest new &quot;smart home&quot; gadgets and I just look like a paranoid kook trying to talk them out of it.

It&#x27;s been a few years I&#x27;ve given up on the idea of privacy with technology.
The number of security flaws that get discovered daily is only the tip of the iceberg.
I&#x27;m pretty sure some governments (or organizations) have had backdoors, be they hardware or software, in place for more than 20 years. We simply don&#x27;t know about it yet (and probably never will). Would that actually be that far-fetched? I think not sadly.
Even the Intel Spectre et Meltdown fiascos are a sign that we have no idea how to actually secure this stuff. And that&#x27;s normal, the very definition of IT security is that nothing can be secure.
Take the whole antiquated concept of processor rings for instance, we are adding a new level every other year now it feels like...
I find it way more interesting (even if it is ultimately &quot;worse&quot;) to adapt to the mentality that &quot;nothing is secure&quot; than &quot;let&#x27;s try and make it secure&quot;, which as stated is in itself a fallacy...

Zilog Z80, 1976&gt; Faggin: Yes, we were concerned about others copying the Z80. So I was trying to figure what we could do that that would be effective, and that’s when I came across an idea that if we use the depletion load the mask that doesn’t leave any trace, then I could create depletion load devices that look like enhancement mode devices. And by doing that we could trick the customer into believing that a certain logic was implemented, when it was not. Then I told Shima, “Shima, this is the idea how to implement traps. Put traps, you know, figure out how to do the worst possible traps that you can imagine,” and then Shima with his mind, that was steel mind, was able to actually figure out a bunch of traps that he could talk about.&gt; Slater: You want to tell us a little about that Shima?&gt; Shima: I didn’t count [on] talking about that mostly. I placed six traps for stopping the copy of the layout by the copy maker. And one transistor was added to existing enhancement transistors. And I added a transistor looks like an enhancement transistor. But if transistors are set to be always on state by the ion implantations, it has a drastic effect on very much. I heard from NEC later the copy maker delayed the announcement of Z80 compatible product for about six months.<a href="http:&#x2F;&#x2F;archive.computerhistory.org&#x2F;resources&#x2F;text&#x2F;Oral_History&#x2F;Zilog_Z80&#x2F;102658073.05.01.pdf" rel="nofollow">http:&#x2F;&#x2F;archive.computerhistory.org&#x2F;resources&#x2F;text&#x2F;Oral_Histo...</a>

While you may find a particular attack if you&#x27;re looking for, in general, it&#x27;s impossible for even the most thorough audits to check for the whole class of such attacks. You&#x27;re not going to look into the chips. Well, you can, but that&#x27;s prohibitively expensive and destructive - even if you could check that this chip was okay, then you still have to throw it out after analysis and plug in a different one.The only feasible thing to do is thorough audits of all the supply chain for every component in your system, ensuring that your supply chain does not include even a single chip from an &quot;untrustworthy&quot; supplier, and even then it reduces the chances of an attack but does not eliminate it.

There are vendors in this space. See: <a href="https:&#x2F;&#x2F;www.harris.com&#x2F;sites&#x2F;default&#x2F;files&#x2F;supplier_counterfeit_awareness_training.pdf" rel="nofollow">https:&#x2F;&#x2F;www.harris.com&#x2F;sites&#x2F;default&#x2F;files&#x2F;supplier_counterf...</a>Good starting point.

How common is it for a seemingly standard security audit to inspect motherboards with such level of detail or at all?They likely needed to have the exact official schematic of the motherboard to compare every single detail of the hardware with.

This article frightens me considering that you can almost always identify the endusers for a lot of the personal devices shipping directly from China.

&gt; The illicit chips could do all this because they were connected to the baseboard management controller, a kind of superchip that administrators use to remotely log in to problematic servers, giving them access to the most sensitive code even on machines that have crashed or are turned off.Obviously this would still be possible without IntelME, but having an always on, highly privileged and remotely accessible baseband definitely makes the modifications easier and smaller...

&gt;Supermicro subcontractorThere is only one I know of: Soyo motherboard

Somewhere poor, where labor is cheap and people are still susceptible to bribes?

&gt;When will this stuff finally have consequences for China?Never, unless hardware manufacturing will take off somewhere else.

Do you really think that another world power would behave differently if they were the factory of the world?

Ha. This looks more like good old capitalism rather than any state sponsored spying.&quot;Nobody&#x27;s making you buy from China&quot;

It’s a cat and mouse game, where escalation can hurt. For example, China might take issue with the systematic compromise of undersea cables.There’s a certain Casablanca-esque “what? There’s gambling happening here?” element here too — Huawei was shipping fake Cisco gear 20 years ago.

&gt; When will this stuff finally have consequences for China?Hahaha funny thing to say after disclosers from NSA&#x2F;CIA&#x2F;FBI&#x2F;GCQH&#x2F;BND...

It already has. Look at the tariffs against them.

When will this stuff finally have consequences for China?Their behavior, not their communication, has been overtly hostile for a while. Yet, very few politicians openly adress the issue.

Because it&#x27;s so awesome. This must be one of this journalist&#x27;s career highlights, to be able to put something like this in a mainstream serious reporting piece.

Why does this keep getting quoted in the comments. Yes we read the article too.

That line in the article made me LOL. Douglas Adams would be proud.

&quot;Two of Elemental’s biggest early clients were the Mormon church, which used the technology to beam sermons to congregations around the world, and the adult film industry, which did not.&quot;

Not to mention that these back doors can be easily disabled, e.g intels high assurance mode, but normal &quot;owners&quot; are specifically not allowed to disable them.The whole thing is a fucking mess.

I&#x27;m pretty certain they do. AMD&#x27;s PSP and Intel&#x27;s ME are really shady and are still enabled in every CPU sold today (and full of disclosed and undisclosed flaws for that matter).

I don&#x27;t think the parent was talking about this specific type of attacks but other types of hardware backdoors. I think the answer to that is very obviously yes, and given what we know about intelligence agencies I&#x27;m even willing to go as far as saying that it is likely (or at least, if you have reasons to be worried about Uncle Sam getting to your stuff you should consider it a very real possibility).Modern ASICs are so complex that I&#x27;m sure that sneaking a tiny backdoor into the behemoth that&#x27;s a modern CPU or embedded SoC would be almost trivial. They&#x27;d also have great plausible deniability in case they&#x27;re found, if something like SPECTRE was an intentional backdoor how would you ever prove it for instance?

&gt; No, it is totally inconceivable AMD and Intel CPUs are backdoored this way.That’s a fairly strong statement.It could be argued it is conceivable as evidenced by the fact we are here talking about it.Also, I’ve read that it’s impossible for any one person to fully comprehend the circuit diagram of a modern CPU. In light of that it seems conceviable any number of wayward circuits could be hidden.

No, it is totally inconceivable AMD and Intel CPUs are backdoored this way. Inserting a microcontroller somewhere on the Ethernet traces and then using IPMI is not as sophisticated as this article wants to describe. Sophistication is necessary from the payload to be stealthy but not the hardware.There are at least two problems with China messing with CPUs: a) they are not made there. TSMC is Taiwan, the Asian parts of Global Foundries are in Singapore, Intel is manufacturing CPUs mostly in the USA with some in Ireland and Israel, only 3DNAND &#x2F; 3DXPoint ICs are made in China. b) the hardware knowledge to change here, well, think of it, the microcontroller on the motherboard is on the few millimetres scale, the parts here are on the few ten nanometres scale so as a rough but not unjust quantifications would say it&#x27;s 100 000 times harder.Whether the platforms are hackable &#x2F; hacked, that&#x27;s harder to say thanks Intel ME and such but the CPUs aren&#x27;t.

Presumably the organisations responsible for hard coding backdoors in chip designs know how to test to confirm their presence.Presumably some adversarial nation-states have moles inside these organisations &gt; know how to remove them prior to fab.Presumably these adversaries export genuine chips to their adversaries, thereby tricking them in to thinking the backdoors made it through the fab process, and only use chips that have the backdoors removed in their own critical infrastructure.Presumably.I’ve always had this fantasy of being a hextuple agent involved in this type of deep espionage.

The US did that in the late 70&#x27;s or early 80&#x27;s to the USSR and it resulted in one of the largest non nuclear explosions in history.

can you be sure that fabs haven&#x27;t been infiltrated and masks changed between design and production in any factory, be it tsmc, samsung, glofo or intel?

And how about the software that runs on these things?

they&#x27;re up front and open that they doalso, they&#x27;re not that hard to dig out<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=_eSAF_qT_FY" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=_eSAF_qT_FY</a>

&gt; One country in particular has an advantage executing this kind of attack: China, which by some estimates makes 75 percent of the world’s mobile phones and 90 percent of its PCs.Intel and AMD are both USA based companies.Is it conceivable their processors contain backdoors in a similar vein?

Wow that’s pretty neat.Terrifying and neat; thanks for the explanation!How easy is it for us (or rather the companies making the hardware) to detect and mitigate this?

If the photo in the article is real you wouldnt be able to identify this component as compromised &quot;just&quot; by visual (even xray augmented) inspection. TVS Diode Array looks the same from the outside, whats more its build in same way with silicon die embedded in tis structure. Other than signal analysis it would take decapping every single component of a motherboard to find this implant.

How specialised? ohmmeters inductance capacitance meters are not exactly exotic.

I don&#x27;t know enough about this, but isn&#x27;t the article saying that the appearance is deceptive:The chips on Elemental servers were designed to be as inconspicuous as possible, according to one person who saw a detailed report prepared for Amazon by its third-party security contractor, as well as a second person who saw digital photos and X-ray images of the chips incorporated into a later report prepared by Amazon’s security team. Gray or off-white in color, they looked more like signal conditioning couplers, another common motherboard component, than microchips, and so they were unlikely to be detectable without specialized equipment.

The chip shown in the article most likely just something journalists found in photo stock by keywords.

Yes, TVS Diode Array (image google to see they look just like in the article) for ESD protection is the obvious candidate, inserted on serial communication lines - perfect spot for signal interception&#x2F;injection.

&quot;But this clever hack is probably not limited to RF and is likely to also be embedded in transformers used for isolating Ethernet lines.&quot;I was thinking along the same lines: a balun could harvest energy and use it for other purposes, but it should either store it for later use using maybe a tiny supercapacitor inside, or inject it into the data stream on the fly not unlike RFID dongles do. Or maybe just have enough power and memory to store a few hundred bytes and alter a few fields here and there, so that for example a network frame coming from a compromised machine can have its source field rewritten as it came from a trusted source.
In theory small pull up resistors also could be swapped with lookalike malicious parts capable of tampering with i2c traffic between boards peripherals so that devices can be activated&#x2F;deactivated no matter what the CPU tells them.We&#x27;re going straight to the point when there won&#x27;t be a single device in the world, from toasters to supercomputers through top brand network gear, that can claim and guarantee to be secure; inserting malicious hardware and firmware is getting just too easy for those with the necessary knowledge and resources. A technically interesting and scary scenario.

The article specifically says that the attack goes after the Baseboard Management Controller (BMC).  In Supermicro equipment this takes the form of a separate SOC made by ASPEED that runs it&#x27;s own operating system entirely separate from the main CPU but with access into the main system through various mechanisms.  Facebook uses the same ASPEED chips in their open hardware projects.

It seems to me that embedding an RF front-end, back-end, and signal manipulation logic in a chip of the size shown in the article would be extraordinarily difficult. Assuming that feat was achievable, it would likely result in serious performance hits on that transmission line right? Signal latency would probably be a dead giveaway for something like EEPROM reads being manipulated.

you would be extremely hard pressed to fit that amount of logic (and code) into that small of a package. and to operate intelligently, as you suggest, it needs a CPU clock, not available to it from where it would sit. And it would need to do signal analysis in real time, with no power and no clock.it&#x27;s not possible.

Looks like I was correct. This image from Bloomberg article seems to indicate SPI flash (<a href="https:&#x2F;&#x2F;twitter.com&#x2F;jamesdotcuff&#x2F;status&#x2F;1048221163607007233" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;jamesdotcuff&#x2F;status&#x2F;1048221163607007233</a>)

It does look like a balun which is clever but based on the pinout I think it is meant to intercept SPI flash and patch it as needed.

So the chip shown in the article looks like a typical SMD balun, it is a type of transformer used to adapt impedance between two transmission line. It’s designed to replace a series a lumped element (capacitor, inductors, resistors) normally used for impedance adaptation (in a T or Pi network). The most common used for the device is directly between an antenna an a RF front-end to serve as an antenna tuner.Technically you could embed and power an RF front-end inside a “flavored” balun to intercept or alter any communication passing through that front-end or even use the antenna to communicate during the down time. So this literally would hack Wifi &#x2F; Bluetooth at low level and inject code and at the same time create a mesh network of malicious devices to relay information. Welcome to IoT Cyberwarfare.But this clever hack is probably not limited to RF and is likely to also be embedded in transformers used for isolating Ethernet lines. Common mode chokes (some SMD chokes also look like the chip they are showing) or even some integrated ESD protection solution would be an ideal target as they are inserted in series of the signal.

I think what is described is an issue with process.If the device is sealed with an anti-tampering system then the contents must be checked by a trusted entity before being sealed.Trying to guess the contents of a box that you cannot open sounds a bit like madness.

&gt; Second, would have it been cheaper to manufacture somewhere more trustworthy (another country?) instead of spending all this time&#x2F;money on your anti-hack systems?I&#x27;d like to know this too. Has the West completely lost the ability to mass produce microchips at even a reasonable cost for financial applications?

You are underestimating the FUN of playing anti-anti-^N-hacks. I have had the privilege to be paid to so anti-anti-^N-hacking on a firewall thingy in the past and it was a challenge and a joy!

This wasn&#x27;t our device. There was a big, reputable company behind the device. We were ordering a number of those and they would be shipped to us directly from China.Also, we were basically locked in due to the magnitude of investment in the software we have developed for the device.Fortunately this only lasted for few months until it was dealt with. It was quite new back then (a decade ago) and it was a surprise for everybody I guess.

When I worked in telecom (a while ago) the manufacturing was shifted from China to Thailand&#x2F;Other SE Asia due to this. The Thai companies weren&#x27;t as efficient, but were much more open and honest when problems would arise., plus they didn&#x27;t blatantly steal tech

First, wow this is both incredible and crazy! Both the China-side hacks and your side&#x27;s anti-hack. Mind. Blown.Second, would have it been cheaper to manufacture somewhere more trustworthy (another country?) instead of spending all this time&#x2F;money on your anti-hack systems?

It&#x27;s mentioned in the article that X-Ray didn&#x27;t help much: &#x27;Gray or off-white in color, they looked more like signal conditioning couplers, another common motherboard component, than microchips, and so they were unlikely to be detectable without specialized equipment&#x27;

Get an x-ray machine? They are surprisingly cheap pieces of hardware if you are willing to deal with a small area, low penetration image. Low penetration means no lead, which makes for something that&#x27;s about as cumbersome as a large bar fridge.

It sounds like they are just ordering stock products from Amazon.

While it may sound sensational this was more of an operational issue, really. We were told by Visa and Mastercard that it is not even a question if we are going to be targeted. If you work in payment card industry you are constantly being attacked and the only way is to make it part of the process to deal with those things. Our network was hacked but what was important was tight, almost mathematical processes around protecting very specific material like credit card data and PINs.For example, PINs are only ever being in unencrypted form inside of Hardware Security Modules and only for the purpose of being encrypted with Visa&#x2F;Mastercard exchange keys. The process was designed so that nobody has enough access to ever get enough cryptographic material to be able to decrypt anything, at least two or three people would have to collude to do anything.It also happens that we put all our resources in investment in software for the platform locking ourselves in. It would be rash decision to change the platform and it would probably kill our company. Also we (correctly) gambled that it would be dealt with quickly.

Look, there are enough supply chain problems with counterfeits already, you don’t want to start thinking of malicious implants

Look, there are enough supply chain problems with counterfeits already, you don’t want to start thinking about malicious implants. Just google for it, it’s massive

Sorry, but this doesn&#x27;t sound as true or there is huge mistakes done in choosing supply chain for such sensitive matter.How come company keep ordering devices from some unverified sources from China, and after hitting a wall keep doing same?How do you accept shipment of such devices without randomly opening and inspecting sample(yes losing all data, but electronic inspection can be done).How you didn&#x27;t investigate that with Visa&#x2F;Mastercard? Whoever does that, he will lose his payment terminal certification after such incident, because they will track them down by IC serials very quickly.What if vendor changed power supply board or even components type on it, and your momentum or weight test will make false positive?Unless... your employer or you buy single devices, on demand, from some shady aliexpress seller. But then, it is plain suicide.

The Chinese operation would simply shut down and reopen under a different name. And the credit card companies are always very worried about their brand image, so they are not interested in any negative publicity.

Why don&#x27;t you guys consider to expose this by suing the bad manufacturer? I believe this could help other truely honest manufacturers both in and outside China to beat the wrong doers.

This is fairly analogous to my comment here: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=18138847" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=18138847</a>Presumably the organisations responsible for hard coding backdoors in chip designs know how to test to confirm their presence.Presumably some adversarial nation-states have moles inside these organisations &gt; know how to remove them prior to fab.
Presumably these adversaries export genuine chips to their adversaries, thereby tricking them in to thinking the backdoors made it through the fab process, and only use chips that have the backdoors removed in their own critical infrastructure.Presumably.I’ve always had this fantasy of being a hextuple agent involved in this type of deep espionage.

The chinese government is probably not interested enough in credit card numbers to warrant involvement.

You can just buy counterfeit anti-tamper stickers but if there is a switch inside the unit that flips a bit in some sort of write-once memory, then that would require removal of an entire chip and replacing it with another that may not be 100% the same. You can have a chain of trust in the system where chips will only talk to each other if they all spit out the right hash. Bury the SPI&#x2F;I2C lines you use for this trust check within the PCB so you can&#x27;t access it without drilling the card and add a layer of anti-tamper traces that trigger another tamper event if disturbed. Now what was just a quick install has turned into a whole PCB rework job where you are having to swap all the chips with a virgin set, assuming you can&#x27;t get your hardware in there prior to final assembly.Use an AES256 key from the factory to hash the chip&#x27;s burned-in serial number and the time from the RTC. Lock out JTAG interfaces so once the chips are burned, they are inside their own fortress. There are a ton of ways to really lock down the hardware other than a shiny sticker and weird screws. Those keep people from breaking their own hardware, anti-tamper tech in chips keep out bad guys.

Payment systems are typically better defended than by just a sticker.It’s not surprising to see a ton of tamper switches, vibration&#x2F;shock sensors, even light sensors. And they’re all powered by an internal batter and separate MCU that will brick the device upon open.

What&#x27;s the motive? If we assume a motive of theft, seems more reasonable that there&#x27;s organized crime involved than a state actor.

&gt; something as simple as an anti-tamper sealTo echo, it&#x27;s actually quite trivial to bypass anti-tamper stickers with acetone and a needle.<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=SqkMIek8sqI" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=SqkMIek8sqI</a>

&gt; as once they were closed they would have anti-tampering mechanism activated so that later it would not be possible to open the device without setting the tamper flagYou didn’t specify what type of anti-tamper was used, but I wanted to jump in and say usually that means nothing. The US government intercepted packages [0] and put in back doors (removing and replacing the seals), so I’m not sure why you were so quick to dismiss state sponsored attacks by something as simple as an anti-tamper seal. You can learn how to do it yourself at most medium to large hacker conferences too (DEFCON, BlackHat, HOPE, and CCC to name a few, but there’s more with it).[0] <a href="https:&#x2F;&#x2F;www.techradar.com&#x2F;news&#x2F;networking&#x2F;routers-storage&#x2F;photos-reveal-nsa-tampered-with-cisco-router-prior-to-export-1249191" rel="nofollow">https:&#x2F;&#x2F;www.techradar.com&#x2F;news&#x2F;networking&#x2F;routers-storage&#x2F;ph...</a>

Maybe a setup to measure inertia tensor and center of mass (in that setup&#x27;s axes) will be easier and I think it&#x27;s what your call &quot;measuring all angular momentums&quot;.

Gosh, I would love to read everything about this. I know there are some videos about anti-tampering card readers on youtube, but not on the feedback race between hackers and security.

A truly sophisticated attacker would leave some easily detectable bad eggs in the stream after countering a countermeasure like the weighing.

Maybe they&#x27;re also measuring centrifugal force for rotations with axes not intersecting the centre of mass.

&gt; We could not measure all possible angular momentums but it was possible to measure one or two that would not be known to the attacker.You only need to measure three angular momentums, all other can be calculated. See <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Moment_of_inertia#Motion_in_space_of_a_rigid_body,_and_the_inertia_matrix" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Moment_of_inertia#Motion_in_sp...</a>&quot;This shows that the inertia matrix can be used to calculate the moment of inertia of a body around any specified rotation axis in the body.&quot;On the attacker side, they only need to make sure three angular momentums match.

I believe you are talking about &quot;Rotational Moment of Inertia&quot; about various axis, instead of &quot;Angular Momentum&quot;.Angular Momentum of a body at rest is zero.

Very interesting methods you used to detect the changes! Out of curiosity: was there a reason that taking an xray of the devices was not an option? Industrial&#x2F;veterinary xray machines can often be had quite cheaply...

Another implication of the parallel axis theorem is that the attacker could perfectly mimic every moment of inertia by shaving plastic. They wouldn&#x27;t have to know which two axes were being tested because there are only three real numbers worth of information in the system to begin with (once center of mass and total mass have been dealt with.) In the whole MOI tensor there are only six free numbers which sounds like a reasonable number of parameters to fix by adding and removing small amounts of material.What was this company doing in hiring an untrustworthy manufacturer to handle secure devices? That&#x27;s playing a game you&#x27;ve lost from the start. Not every problem is technical!

Ah, you are right. English is not my native language.

Indeed there are only a finite number of moments. If this was not true you could effectively 3d scan an object by just measuring its inertia response.

I enjoyed your comment.  But you are looking at the difference in the inertia tensor of an adulterated board from a non-adulterated one.  Not the inertia tensor itself.  Signal to noise problems with the measurement.

&gt; We could not measure all possible angular momentums but it was possible to measure one or two that would not be known to the attacker.You mean moment of inertia, not angular momentum.You could measure all of them! Given the moments for the three principal axes at any point, you can use the parallel axis theorem to calculate all the rest. In general, there are 10 degrees of freedom: 3 for the position of the center of mass, 3 for the axes, three for the moments, and for the the total mass.For a nicer way to count and to do the math, you have the inertia tensor at the CM (a 3-dimensional rank-2 symmetric tensor, 6 DoF) plus the location of the CM and the mass.In any event, this is a cute tampering-detection trick, but I would have started with an X-ray or CT scan.

Ah, yes, and then we would voluntarily go out of business. And the supplier will just have another customer that would not be so principled.

I&#x27;m sure the Chinese government would be willing to keep afloat companies that allow them to tamper with.

Could this be solved by a clause in the contract that specified if any randomly sampled devices came tampered with, the manufacturer did not get paid?

At what point does it get cheaper to buy from sources that are more expensive but more trustworthy?

I know it&#x27;s nice to blame China for everything... but it&#x27;s not really the root of the problem here, supply chain management and control is.

Maybe time to buy your financial hardware from somewhere not china?

The main lesson I learned from my Dad’s employer (twice) was always have two vendors. You can play them off of each other. When I did contract work I saw how powerful getting out or putting for away the checkbook can be. Large vendors ignore you if the checkbook isn’t moving.I say “twice” because they were also the biggest employer in town. I got out of there. Slim pickins for career opportunities that didn’t revolve around BigCo.

So you had hard evidence that your manufacturer was vandalising your property? Couldn&#x27;t you drop them?

It&#x27;s hard to compete domestically against low prices caused by China&#x27;s completely different standards for wages and human rights for laborers. Plus, here we don&#x27;t like the idea of manufacturing industries being subsidized by the government (except in the case of &quot;defense&quot; of course...), while China obviously has no qualms with doing so.

Counterpoint: even if we ignored the fact that you cannot possibly produce the volumes of chips necessary at the price necessary in your country rather than in &quot;we don&#x27;t have to acknowledge all the human rights violations&quot; countries, why would you believe this problem goes away if chip manufacturing were done in your own country, rather than another?The moment the option of taking control of a production line of something _this important_ becomes available, your local specialized organized crime outfits will start to figure out ways to insert themselves into those production lines, learning the ins and outs, and figuring out a way to get something, anything, in there that won&#x27;t be noticed but will give them a hook into millions of systems.The law does not prevent crime. It just puts a price on it. While that price is typically too high for individuals, for organizations that have no business registration to revoke, and no CEO to drag to court, it is an entirely trivial cost.

Canadian steel is considered by this administration to be a national security risk. But Chinese made boards and chips installed in weapons systems and crucial data centers? No problem.Let that sink in for a moment.<a href="https:&#x2F;&#x2F;www.wsj.com&#x2F;articles&#x2F;dont-trust-the-chinese-to-make-microchips-for-the-military-1509059023" rel="nofollow">https:&#x2F;&#x2F;www.wsj.com&#x2F;articles&#x2F;dont-trust-the-chinese-to-make-...</a>

&gt;Seriously, why are we still outsourcing chip manufacturing to other countries?I actually agree it might be better for the Americans to manufacture things in America - especially things used in critical government systems.But this seems like a human problem - if all the factories were moved to America, couldn&#x27;t those factory managers etc also be bribed?American manufacturing in the 60s was rife with unions w&#x2F; ties to organized crime.

&gt; Sure it&#x27;s cheaperThat&#x27;s enough, full stop, say no more. The other costs are real yet they&#x27;re either not marginal, are borne by others, or both.

You&#x27;re conflating a few things there. Whilst companies like to make noises occasionally about saving the planet, doing the right thing, making (your country goes here) great etc, it&#x27;s just horseshit. They exist to make money. That&#x27;s it.

Seriously, why are we still outsourcing chip manufacturing to other countries? Sure it&#x27;s cheaper, but we sacrifice a lot to have a society of corporate slaves build our tech. Security, core domain knowledge, capability, corporate secrets, patent rewards and enforcement, etc... All of it you throw away the minute you ship your manufacturing out of the country. I&#x27;ve seen enough board printing machines out there to start working on our own. As a country, we need to close this gap, more automation and capability and there will be no need to outsource circuit board printing and manufacturing. We will be much better off.

x-ray would have been easier, as others have said.but you&#x27;re talking about the addition of an entire board! probably on the order of 10% of the size of the main boards.in this article, perhaps dumbed down or altered, they are talking about the addition of a single, tiny chip, too small to even be an MCU let alone have wireless capability (which BTW requires an antenna).

Americans can be bribed to do anything. However, you don&#x27;t really need to bribe or ask someone to betray their country and fellow citizens when it&#x27;s done on Chinese soil...

The bad actors could be brought to trial in a United States court, which is a level of deterrent not included in offshore manufacturing. If a US manufacturer was found selling tampered chips then the company itself could be held liable. This would create a general pressure to increase onsight security.There are other benefits to us based manufacturing, but you only brought up the crime aspect so I will leave the other benefits unsaid.

It&#x27;s possible, but it&#x27;s much, much, much easier for an American company to hold another American company accountable when something like this happens. Instead of having to go through all those hoops, you sue the pants off the manufacturer.

For the folks commenting below that we should bring the manufacturing back to the US, why wouldn&#x27;t the bad guys just start bribing American workers to insert the attack hardware into devices made here?It&#x27;s not like Americans are somehow above being bribed.

There were other considerations like the fact we were actually buing it from large reputable company and what happened was that some employees were doing it with no involvement of the company.The fact is, doing any kind of hardware production in China, you have to be aware Chineese have different value system and you would not be suited doing any business if you throw tantrum at any sign of apparent dishonesty (assuming the company was involved which they could not have been as they have been the ones damaged the most).If the company does screw you (like replacing components for something cheaper) they typically will not be thinking they are doing anything wrong. They are just testing if you notice and if you do not they will say it makes no difference for you but saves them costs.The way to work is then verify everything and politely point it out. If you notice they will correct apparent mistake.

This is also such a good anonymous story that I&#x27;d give decent odds it&#x27;s made up.

Probably the cost per unit is too attractive

he&#x27;s saying that his company knowingly allows a percentage of fraud, identity, and financial theft to occur against his customers&#x27; customers because there are enough layers of indirection where they (he and his bosses) probably can&#x27;t be held personally responsible. Enough efforts (fancy tables with big receipts, etc) have been provided for coverage in case of a court case or media blitz but actually using a reputable supplier is clearly too expensive.I can&#x27;t believe the parent comment is so brazen. It makes me physically sick.

Maybe you and me are missing something here. It seems crazy that somebody would go to these lengths without switching suppliers, there must be an underlying reason that is assumed to be understood by the informed reader. (But isn&#x27;t understood by me and you it seems.)

This problem is not specific to China, it&#x27;s a problem of going with the lowest priced supplier.  Your proposed recourse presumes that the supplier is making enough money on the deal to engage in a contract with teeth... many are not.  Also, even if they moved everything to the U.S. they&#x27;d likely have different flavors of the same types of issues (i.e. did the NSA embed someone in the company to implant backdoors etc.)  Fraud, sabotage and espionage aren&#x27;t confined to specific countries.  When greed drives you to move operations to the absolute cheapest places you can find on Earth, that&#x27;s not going to be without its own set of risks and problems.

Honestly, to anyone bashing GP, look into your pockets, laptops, watches, cars, TVs, routers, CCs, singing toys, bitcoin mining ASICs. All with sealed black-box chips. Sure we can x-ray a couple of randomly stripped chips, but each one?Now show me alternatives when most consumer grade electronic parts are fully or partially made in China.

Wait a minute... So your company has a Chinese equipment supplier, finds out that the supplier is tampering with your purchased equipment, and your solution is to add criteria to the incoming inspection?No wonder China keeps screwing with you guys. You aren&#x27;t supposed to eat that cost! Write a PO with tons of fine print that says &quot;We will disassembly units at random for compliance inspection. Non compliant products will be returned at the suppliers expense.&quot; And then add a clause that says &quot;&gt;3 non-compliance events in under # months will result in the entire PO (10 or 20 units) being returned and all contracts cancelled.&quot;I cannot believe you are getting screwed by a company you choose to do business with and yet you eat cost to ensure they aren&#x27;t screwing you. Just get a new supplier! Do on-site inspections at their facility. This is nuts.

Now I’m picturing a 300lbs photocopier being spun on three axes. How much tape would it take to keep doors and parts from flying off?

is measuring the moment of inertia not standard? I think any mechanical engineer would suggest it for that kind of problem (disclaimer; I&#x27;ve suggested it before for the same problem)

This reminds me of a story about, IIRC, Soviet intelligence personnel determining that a photocopy machine at their consulate in the USA was bugged by the CIA by measuring it&#x27;s weight and comparing that value to the standard value published by Xerox.Modern times, but same old methods of &quot;debugging&quot;The angular momentum stuff is innovative though.

&gt;measuring angular momentum on a special fixtureCan you elaborate on this concept a bit? I&#x27;m not familiar with this term and the sources I looked up were pretty physics-y and out of my depth.This sounds like a very interesting and creative solution. Good lateral thinking on your part! :)

The better question is how did we stop manufacturing.

This feels cargo cultish. Products drop from the sky. One day they become poisonous. You have no idea how to reproduce them locally. So you come up with hacks to make then less dangerous.We really need to get back into manufacturing if this is our brave new world.

What good is angular momentum when the producer can have fluctuations in its supply chain? Yes you can see when devices are not the same, but what if that happens all the time, legitimately?

I do not for one minute believe, that china, whose production floors are spotless, watched like a hawk and every product tested, did not know this was being added. They would have been able to tell themselves by weight. Yes, product weight is used for shipping manifests too.This is obfuscation of the fact, that china as a state actor has perpetrated this crime against our country period. full stop.

Thanks for sharing this story, and I hope you aren&#x27;t put off by the huge thread of people second-guessing your competence. We need more of your kind of story.

Was the secret service contacted? Card skimmers are a big no no. Family friend works for Dept of weights and measures and finds skimmers all the time on gas pumps. Scary stuff.

I have worked in card payment industry. We would be getting products from China with added boards to beam credit card information. This wasn&#x27;t state-sponsored attack. Devices were modified while on production line (most likely by bribed employees) as once they were closed they would have anti-tampering mechanism activated so that later it would not be possible to open the device without setting the tamper flag.Once this was noticed we started weighing the terminals because we could not open the devices (once opened they become useless).They have learned of this so they started scraping non-essential plastic from inside the device to offset the weight of the added board.We have ended up measuring angular momentum on a special fixture. There are very expensive laboratory tables to measure angular momentum. I have created a fixture where the device could be placed in two separate positions. The theory is that if the weight and all possible angular momentums match then the devices have to be identical. We could not measure all possible angular momentums but it was possible to measure one or two that would not be known to the attacker.

It could be interpreted that way, but this sort of activity isnt novel or unknown.The Feds run key hardware through third parties to detect counterfeit hardware. The “new” thing is the targeting of devices headed for commercial customers.

These chips were REPORTED TO BE designed by the Chinese military and inserted by PLA agents.Big difference, which is parent&#x27;s point.

Anti-China propaganda? These chips were designed by the Chinese military and inserted by PLA agents. Do you have to start calling things propaganda to make it seem like this is baseless criticism of China to those who have not read the article?

&gt; Could you think of any other sovereign power who would want a backdoor into servers used by billions of people across the internet?I cannot think of any sovereign power that wouldn&#x27;t.

This comment could easily be interpreted as pro-China propaganda.

&gt; Could you think of any other sovereign power who would want a backdoor into servers used by billions of people across the internet?The US&#x27;s NSA would want and probably does have such a thing.

It seems like many people take communist-like propaganda at face value.The Chinese government would like to have almost-complete control. The Chinese government publicly says it has high levels of control.But when they can&#x27;t effectively regulate their medical (mass HIV infection from blood plasma needle reuse), food (tainted milk), or chemical (unlicensed mass CFC production) industries... reality seems to differ.

your comment looks like an pro-china propaganda. In china the state has heavy control on all the companies. The hardware is manufactured in china. I don&#x27;t think something like this is happen without the state&#x27;s knowledge.

The United States is having a trade war with China partially because of this kind of thing.

This story could easily be interpreted as anti-China propaganda. Could you think of any other sovereign power who would want a backdoor into servers used by billions of people across the internet? Hardware comes from China. This doesn&#x27;t mean that the Chinese government orchestrated the attack. The United States government is having a trade war with China. This article&#x27;s publication isn&#x27;t just coincidence.Further, those with insight about how to use the backdoor definitely made use of them. This is one of the central issues with government-mandated backdoors. They are introduced for one group but used by others.

In retrospect it seems that the decision to move all manufacturing to China was ill-advised.Western companies ultimately will have no choice than to move it all back. (And Trump will want to take credit for that.)I can understand all the big guys denying this. It&#x27;s very hard to fix and very bad for business.

&quot;In one case, the malicious chips were thin enough that they’d been embedded between the layers of fiberglass onto which the other components were attached, according to one person who saw pictures of the chips.&quot;Wow.

they have literally every reason to deny and literally no reason to say it&#x27;s true

This article is more or less total bullshit. At _best_ that device might be a mechanism to cause failure intentionally. And there are tons of ways to detect it with commodity technology, and plenty of vendors who implement that technology for assembly manufactures commercially.

&gt; I wonder what reason Apple has to hide.The perception is that Apple is perfect and worth paying 3x the cost?EDIT: Curious if all of these Apple comments are going to disappear. I believe they have a strong marketing team to hide dissent.

Assuming Bloomberg&#x27;s story is true, I wonder what reason Apple has to hide. Not wanting to upset relations with the PRC govt?

Statements from Amazon, Apple, Supermicro and Chinese government.<a href="https:&#x2F;&#x2F;www.bloomberg.com&#x2F;news&#x2F;articles&#x2F;2018-10-04&#x2F;the-big-hack-amazon-apple-supermicro-and-beijing-respond" rel="nofollow">https:&#x2F;&#x2F;www.bloomberg.com&#x2F;news&#x2F;articles&#x2F;2018-10-04&#x2F;the-big-h...</a>From Apple:&quot;Over the course of the past year, Bloomberg has contacted us multiple times with claims, sometimes vague and sometimes elaborate, of an alleged security incident at Apple. Each time, we have conducted rigorous internal investigations based on their inquiries and each time we have found absolutely no evidence to support any of them. We have repeatedly and consistently offered factual responses, on the record, refuting virtually every aspect of Bloomberg’s story relating to Apple.&quot;

The Big Hack: How China Used a Tiny Chip to Infiltrate Amazon and Apple