Forty+ different AI chips

What are we going to do with 40+ different AI chips?

This week, the team looks at AI chips again, this time motivated by an article in EE Times about once such chip, Graphcore, and touts it as “the most complex processor” ever at some 20 billion transistors. The VC-backed company out of Bristol, UK is also valued on paper at $1.7b, gaining it the coveted “unicorn” status, apparently the “only western semi-conductor unicorn”.

This being one of 40+ such AI chips (and that may be conservative), the odds of success are tough and the task formidable. But even if only 2 or 3 of such chips become successful, that’s already a significant disruption to the market.

The Graphcore chip is 16nm, 1.6GHz, and comes in a PCIe card at 300W. You can stack 8 of these in a 4U chassis, so 2.4 kW just for those.

After a mini-rant about respected publications succumbing to clickbaits, the team talks about how cooling will be an issue and calls again for more clarity in performance metrics since the chip is rated at 125 TFlops but we don’t know at what precision. Shahin reminds the team of his suggestion to clarify things by including precision in the metric, like DFlops for double precision, and then S for single, H for half, and Q for quarter precision.

Henry talks about how hard it is to build and test complex software like this despite Shahin’s view that the modern software stack is too high so the chip need only be concerned with a couple of layers, codes are new and open to getting recompiled, it’s increasingly open source, cloud providers and large customers have the wherewithal to do the job, and traditional HPC customers have the willingness to do the work if performance enhancements are there.

No “Catch of the Week” this time since Henry had a hard stop. We’re used to it!

Listen in to hear the full conversation.

Download the MP3 * Subscribe on iTunes * RSS Feed

Sign up for our insideHPC Newsletter

Amdahl’s Law and GPUs, Asian Student Cluster Competition

Results of the Asian Student Cluster Competition

In this episode, Dan has just come back from China and reviews the results of the Asian Student Cluster Competition and HPC workshop.
For the first time, a non-mainland-Chinese team wins the top spot. Taiwan takes the gold in part by their stellar performance in HPCG benchmark where they achieved 2 TFlops, some 25% better than the 2nd best team. The system was a 5-node cluster with Infiniband FDR interconnect. Other interesting info is shared on various codes and configurations.

GPUs and Amdahl’s Law

Dan also mentions that reports from some of the TOP500 sites suggest that GPUs are doing 93-97% of the computation. This sounds very impressive but Shahin points out that since GPUs have hundreds of cores, they should be doing much better, that 93-97% is in fact not as good as it should be at that scale of system and problem size. He is still waiting for some actual utilization data on GPUs too.

Catch of the Week

Henry:

Henry points out many security cameras, offered by several brands but are all manufactured by the same vendor back in China, have big time vulnerabilities so he’s staying away from all of them until further notice. Shahin wonders why they are called “security” cameras!

P2P Weakness Exposes Millions of IoT Devices

A peer-to-peer (P2P) communications technology built into millions of security cameras and other consumer electronics includes several critical security flaws that expose the devices to eavesdropping, credential theft and remote compromise, new research has found.

Shahin:

Shahin talks about Jaguar-Land Rover planning to offer a cryptocurrency wallet to reward drivers that participate in providing traffic and other types of data. He likes their catch phrase: zero emission, zero accident, zero congestion.

Drivers will be able to earn cryptocurrency and make payments on the move using innovative connected car services being tested by Jaguar Land Rover.

 

Dan:

Dan laments the confiscation of his external camera battery at the airport in China because the spec label was a little worn off and the authorities could not read it to ascertain its safety despite his willingness to get a note from the airline, etc.  Nice expensive battery, but at a medium-sized paperback book, maybe following the rules strictly is not a bad idea.

Listen in to hear the full conversation.

Download the MP3 * Subscribe on iTunes * RSS Feed

Sign up for our insideHPC Newsletter

In-Memory, Caching, Persistence

Asian Student Cluster Competition

The ever-popular “calendar segment” is back (yes, that is us being sarcastic) as Dan describes his upcoming trip to Dalian, China for the Asian Student Cluster competition. 20 teams are expected to compete, mostly from mainland China, but also from Europe, South America and other parts of Asia. All of them have gone through a rigorous qualifying process. Chinese vendor Inspur provides all the equipment (except for accelerators, which remain the responsibility of the teams) based on the configuration that the teams request.

As always, the team, and the rest of humanity, is highly supportive of the student cluster tournaments that take place at SC, ISC, and this Asian one, in addition to many regional supporting events.

Dan also lets us know how to go about picking the right seat in the airplane. There’s an app for that!

New Optane

Intel’s new Optane is M.2 format with 16GB or 32GB of Opatne and 256GB, 512GB, or 1TB of Flash backing it. This made Henry think of history of caching and how people used to go through laborious data placement on outer cylinders of hard disks. Then you had DIMMs with Flash behind it. The discussion goes to the algorithms and policies that manage data movement, which is another form of optimizing workload management, and the different combinations of fast/small/expensive capacity backed up with slower/larger/cheaper capacity. And how persistence impacts this equation and to what extent applications may want to optimize around these. Then there is in-memory processing, ram-disk, and how in the future, the ever-increasing size of memory can make those standard practice.

Catch of the Week

Henry:

Henry is catchless but we can expect a healthy debate. Shahin produces two to make up for it.

Shahin:

Shahin has new reinforcement that IoT and HPC will converge in interesting ways. 1) We all know that IoT is the fountain of data and will generates so much data that you need HPC (and AI) to make sense of them, but 2) what is also very interesting Digital twins will need to simulate and predict the behavior of the real thing.

What we mean when we talk about digital twins

“Watching it prompted me to wonder how much data it takes to create a digital twin. If I had a digital simulacrum of a machine and could apply different environmental or mechanical factors to it, how large could that original simulacrum be? It turns out that’s not how digital twins work. They aren’t virtual doppelgangers. They are actually a series of algorithms that connote how machine moves or behaves. In other words, a twin isn’t a twin so much as it’s…math.”

 

Amazon Workers Are Listening to What You Tell Alexa

“A global team reviews audio clips in an effort to help the voice-activated assistant respond to commands. The team comprises a mix of contractors and full-time Amazon employees who work in outposts from Boston to Costa Rica, India and Romania, according to the people, who signed nondisclosure agreements barring them from speaking publicly about the program. They work nine hours a day, with each reviewer parsing as many as 1,000 audio clips per shift, according to two workers based at Amazon’s Bucharest office, which takes up the top three floors of the Globalworth building in the Romanian capital’s up-and-coming Pipera district. The modern facility stands out amid the crumbling infrastructure and bears no exterior sign advertising Amazon’s presence.”

Shahin thinks it’s as simple as supervised learning needing supervisors, so the question is the legal framework and jurisdiction issues, and social policy, and not technology.

Dan:

It is time to panic more as Dan shares the story of the Chinese scientist who’s presumably blended monkey and human genes. It’s Planet of the Apes all over again.

Chines Scientists Gene-Hacked Super Smart Human-Monkey Hybrids

“For the first time, scientists have used gene-editing techniques to make monkey brains more humanlike. The monkeys, rhesus macaques, got smarter — they had superior memories to unaltered monkeys, according to recently-published research that’s kicked off a fiery debate among ethicists about how far scientists should be able to take genetic experimentation.”

Listen in to hear the full conversation.

Download the MP3 * Subscribe on iTunes * RSS Feed

Sign up for our insideHPC Newsletter

Black Hole Seen, with Data to Match

Here’s this week’s synopsis.

Black Holes Visualized

The news of the cool visualization of an actual black hole leads to interesting issues in HPC land. Shahin is at pains to give credit where it is due while considering it as an achievement in data visualization not unlike many others before it. Yes, it’s about a fascinating topic, but that’s also not unlike many others in the past.
But the team moves on to the real point: the size of the radio data that had to be collected and managed and processed to visualize it. 1.75 PB of raw data from each telescope meant a lot of physical drives that had to be flown to the data center. Henry leads a discussion about the race between bandwidth and data size, various companies’ plans to launch thousands of satellites to help get away from sneakernet, and the imminent arrival of 5G. We’ve discussed large scale data movement in previous episodes and think it’s an important issue for HPC, AI, and Cloud.

Catch of the Week

Henry:

That sneakernet discussion above is it for Henry this week.

Shahin:

Mapping Space Debris (video)

LeoLabs is a company that maps objects in the low Earth orbit (LEO). The visdeo shows actual trajectories of 12,401 low Earth objects in space being tracked on August 24, 2017 by LeoLabs’ phased array radars. Video loop shows approx 2 hours of data.

Dan:

Scientists put human gene into monkeys to make them smarter, human-like

Making monkeys more smart and human-like, scientists have used gene-editing to insert human brain gene in a monkey.

For the first time, a team of Chinese scientists made use of gene-editing techniques to make monkey brains more human-like. By the end, the monkeys, rhesus macaques, got smarter and had superior memories as compared to the unaltered monkeys.

The team doubts this is a true story and that leads Shahin to his first rant on the show when he complains about previously reputable publications succumbing to clickbait.

We’re More Likely Than Not Living In A Computer Simulation, MIT Professor Suggests

An MIT professor has said he believes it’s “more likely than not” that we are living in some kind of simulated universe, given that we ourselves are not far away from being capable of creating hyper-realistic simulations ourselves.

Yet another story that raises eyebrows. This one leads the RFHPC team to create a new award on the spot!

Listen in to hear the full conversation.

Download the MP3 * Subscribe on iTunes * RSS Feed

Sign up for our insideHPC Newsletter

Enterprises go HPC, Chips go Open Source, China goes for the top spot

We continue to want to make these introductions pretty brief here but not this time, apparently! Here’s this week’s synopsis.

Nvidia GTC 2019 announcements

We discussed the recent GTC conference. Dan has been attending since well before it became the big and important conference that it is today. We get a quick update on what was covered: the long keynote, automotive and robotics, the Mellanox acquisition, how a growing fraction of enterprise applications will be AI.

In agreement with the message from GTC, Shahin re-iterates his long-held belief that the future of enterprise applications will be HPC and once again asserts that AI as we know it today is a subset of HPC. Not everyone agrees. Henry brings up varying precisions in AI and a discussion ensues about what is HPC. There seems to be agreement that regardless of what label you put on it, it is the same (HPC) industry and community that is driving this new trend. And that led to a discussion of selling into the enterprise and the need for new models and vocabulary and such.

Speaking of varying precision, there is also Nvidia’s new automatic mixed precision capability for Tensorflow and there is a bit of discussion on that.

China plans multibillion dollar investment in supercomputing

On the heels of the Aurora announcement, there was news in the South China Morning Post that the top spot in supercomputing is something the country is investing in. No surprise, but interesting to see, and consistent with the general view that supercomputing drives competitive strength.

Catch of the Week

Henry:

Facebook Stored Hundreds of Millions of User Passwords in Plain Text for Years

Hundreds of millions of Facebook users had their account passwords stored in plain text and searchable by thousands of Facebook employees — in some cases going back to 2012, KrebsOnSecurity has learned. Facebook says an ongoing investigation has so far found no indication that employees have abused access to this data.

Shahin:

MIPS R6 Architecture Now Available for Open Use

MIPS 32-bit and 64-bit architecture – the most recent version, release 6 – will become available Thursday (March 28) for anyone to download at MIPS Open web page. Under the MIPS Open program, participants have full access to the MIPS R6 architecture free of charge – with no licensing or royalty fees.

Dan:

Vengeful sacked IT bod destroyed ex-employer’s AWS cloud accounts. Now he’ll spent rest of 2019 in the clink

An irate sacked techie who rampaged through his former employer’s AWS accounts with a purloined login, nuking 23 servers and triggering a wave of redundancies, has been jailed.

 

Dead LAN’s hand: IT staff ‘locked out’ of data center’s core switch after the only bloke who could log into it dies

‘We can replace it but we have no idea what the config is on the device’

Listen in to hear the full conversation.

Download the MP3 * Subscribe on iTunes * RSS Feed

Sign up for our insideHPC Newsletter

Multicore Scaling Slow Down, and Fooling AI

The team has an animated discussion about multicore scaling, how easy it seems to be to mislead AI systems, and some good sized catches of the week. A common thread is “data” as is often the case these days.

Dan makes a couple of important announcements.

First is the idea that is brewing about revamping the podcast.

Second, to the dismay of the vast number of his supporters, is his decision to not run for the highest office in 2020!

We continue with making these introductions pretty brief here. This time, we include not only the links but also the first paragraph of the linked page as a block quote so you have a bit more information about what is discussed.

Specialized Chips Won’t Save Us From Impending ‘Accelerator Wall’

As CPU performance improvements have slowed down, we’ve seen the semiconductor industry move towards accelerator cards to provide dramatically better results. Nvidia has been a major beneficiary of this shift, but it’s part of the same trend driving research into neural network accelerators, FPGAs, and products like Google’s TPU. These accelerators have delivered tremendous performance boosts in recent years, raising hopes that they present a path forward, even as Moore’s law scaling runs out. A new paper suggests this may be less true than many would like.

 

Nice ‘AI solution’ you’ve bought yourself there. Not deploying it direct to users, right? Here’s why maybe you shouldn’t

Top tip: Ask your vendor what it plans to do about adversarial examples.

RSA It’s trivial to trick neural networks into making completely incorrect decisions, just by feeding them dodgy input data, and there are no foolproof ways to avoid this, a Googler warned today.

 

Catch of the Week

MyEquifax.com Bypasses Credit Freeze PIN

Most people who have frozen their credit files with Equifax have been issued a numeric Personal Identification Number (PIN) which is supposed to be required before a freeze can be lifted or thawed. Unfortunately, if you don’t already have an account at the credit bureau’s new myEquifax portal, it may be simple for identity thieves to lift an existing credit freeze at Equifax and bypass the PIN armed with little more than your, name, Social Security number and birthday.

 

Announcing the Open Sourcing of Windows Calculator

Today, we’re excited to announce that we are open sourcing Windows Calculator on GitHub under the MIT License. This includes the source code, build system, unit tests, and product roadmap. Our goal is to build an even better user experience in partnership with the community. We are encouraging your fresh perspectives and increased participation to help define the future of Calculator.

 

Huawei Sues The US, Prodding It to Prove Suspicions

THE WORLD’S LARGEST telecommunications-equipment company, China’s Huawei, is suing the US government. But the suit isn’t just about US law. It’s part of Huawei’s larger campaign to defend its role as a global provider of telecom gear amid fears that its technology is or could be used by the Chinese government for spying. In essence, Huawei is challenging the US government to prove its suspicions.

Listen in to hear the full conversation.

Download the MP3 * Subscribe on iTunes * RSS Feed

Sign up for our insideHPC Newsletter

AI: Realness and Bias

Starting with this episode, we’ll get a bit more efficient in describing the episodes. Please let us know if you prefer the long format. If you just subscribe on iTunes and never see these words, well, that tells us something too!

In this episode, the team discusses AI, bias in AI, and just how real actual AI out there is. Ethics in AI, policy, legal framework are all big threads here. The trigger is the rather funny article Artificial Intelligence, You Know it isn’t real, yeah?

Catch of the Week

Shahin applauds NIST’s new Risk Management Framework, and especially the inclusion of supply chain security, something he and Henry keep bringing up.

Henry discusses sensationalism in technical coverage by the example of an article that says blockchains can be hacked but lacks enough depth and thus fails to impress. As expected, Shahin comes to the defense of the technology, explaining that it depends on the consensus algorithm and participation, etc. not just blockchain per se. Discussion ensues about all manner of blockchains and the spectrum that is forming there with permissioned and permissionless chains.

Dan: In a switch from uplifting news to scary ones, Dan shares the news that Kalashnikov rolls out a weaponized suicide drone.

Listen in to hear the full conversation.

Download the MP3 * Subscribe on iTunes * RSS Feed

Sign up for our insideHPC Newsletter

Nvidia, Mellanox: Married!

Big news in the industry today was Nvidia buying Mellanox for $6.9B. This called fo an emergency session of our crack panel.

While it will be several months before the full impact of this merger is felt, the RFHPC team believes this will change both the HPC and the Datacenter markets. It also signals Nvidia’s journey towards becoming more of a systems company and gives them a better shot at the enterprise AI market.

This is also good news for all the alternatives in the market, Shahin and Henry believe. There are a large number of AI chips in the works around the globe, and a growing number of interconnect options on the market. They will now have a chance to present themselves as a more neutral option.

Since the combined company will now represent a bigger portion of the total bill, it has a strengthened hand in the face of growing competition, while, on the other hand, becoming a more visible part of the total system cost, inviting new competition.

Listen in to hear the full conversation.

Download the MP3 * Subscribe on iTunes * RSS Feed

Sign up for our insideHPC Newsletter

Arm for Exascale is Coming

The show starts with a brief reference to Henry’s “Gator” nickname, a shout out to listeners 13, 14, and 15, plus a bad Arm processor pun. (Puns are the lowest form of humor other than limericks, Dan wits.)

This is an Arm heavy show, with our opening discussion concerning the Arm system at Sandia and a talk given by Sandia’s Michael Aguilar at the recent HPC Advisory Council’s Stanford Conference. The new system, dubbed Astra, was built by HPE and is the biggest Arm-based super on the Top500 list at 2.3 PFlop/s.

The guys discuss how quickly the system was brought up and how Sandia didn’t run into any major problems along the way – which is unusual for a system utilizing a new processor. We take a tangent into a discussion of new chip architectures and how this is leading to more options for customers.

Keeping with the Arm theme, the conversation moves to the new Arm Neoverse 128 core server processor. The guys are a bit agog over the 7nm size of the processor, wondering who is fabbing the chip, guessing TSMC. The new chip is 2.5x faster than previous Arm server processors and, according to Arm, also uses 30% less power.

The conversation moves to RISC V and whether it will be used as an accelerator or a CPU – eventually agreeing that it can be both.  We discuss how the chip can be used in various ways and how it can potentially replace a lot of things, including ASICs, which is pretty mind blowing.

Catch of the Week

Henry’s Catch of the Week concerns a new hardware hack that allows miscreants to capture payment info from a phone at the gas station. The bad guy uses a Bluetooth based skimmer to send payment info from contactless payment cards (or phones assumedly) via SMS message to the miscreant. You can read the frightening details at the link above, which goes to Krebs on Security – a great site if you want to scare yourself senseless.

Shahin chimes in with something even scarier – the Evil USB Cable:  a USB patch cable that has an embedded wifi transmitter that can send all of the data flowing through that cable to a bad guy. Yikes!

Dan attempts to put minds at ease by exposing the truth behind a hacking myth:  can a hacker easily get control of your laptop’s webcam? The answer? Nope, they can’t. A Wall Street Journal writer worked with a highly qualified white hat hacker to see just what it would take for a hacker to gain control of a Windows or Mac embedded camera.

It turns out that penetrating a laptop camera is pretty difficult and not really possible unless the user cooperates to make it work. On the Windows side, the writer had to disable Microsoft’s anti-virus and real time virus checking in order to get the hacker payload into her system. The file was also flagged as dangerous by Microsoft Word, so she had to dismiss that warning as well.

The Mac OS was even more difficult for the hacker to penetrate. First, the user had to install LibreOffice, meaning she had to disable Mac security settings that prevent unverified software from installing on her system. She also had to disable the security inside LibreOffice.

Take a look at the article and see if you agree with Dan, who believes that laptop cameras can’t be hacked by outsiders unless you essentially invite them in by disabling your OS and application security.

Subscribe on iTunes or Download this week’s edition of Radio Free HPC for a chance of winning out eternal gratitude and respect!

Download the MP3 * Subscribe on iTunes * RSS Feed

Sign up for our insideHPC Newsletter

The sinking of the Itanic, a respectful farewell

Our conversation begins with Dan berating Henry for cheaping out on a new headset for our Radio Free HPC recordings. (Henry has since relented and pried open his wallet to purchase a truly fine headset.)

Finally, we get on to the show proper. After years of futility, Intel has finally put a stake in the heart of their Itanium processor. The final shipment will take place in 2021. The boys discuss some Itanium history and reasons behind the end of the processor. Shahin gives us a tutorial on the history of 64 bit computing and we discuss the chip wars in general.

Following a respectful farewell to Itanium and all that it offered, we moved on to discuss how the European exascale effort is shaping up. Recent news articles are discussing how countries in the European Union are ganging up in an effort to win the honor of hosting the fastest supercomputer in Europe. The first consortium consists of Nordic countries Finland, Denmark, Norway, Switzerland, plus the Czech Republic, Belgium, and The Netherlands. Two of the advantages these countries have are power costs that are half of the European average, along with temperatures to match (which will help with cooling).

The guys talk about what the term ‘pre-exascale’ really means. Is it a 100 PB system? 200 PB? 300 PB? Tune into the broadcast to hear the thrilling answer. We also discuss the composition of the machine in terms of processors and accelerators.

Catch of the Week

Henry’s Catch of the Week confirms his distain for all things crypto currency related. A Canadian bitcoin exchange president dies suddenly and takes his password to his grave, taking with him the coins of more than 100,000 users said to total more than $130 million US. Henry is vindicated and Dan heartily agrees with him and piles on with “I hope Bitcoin goes below zero.” Shahin defends Bitcoin and Blockchain in general.

Shahin shares a game called The Last Man, where people compete to become the last person to learn about an event, such as who won the Superbowl.

Dan’s Catch of the Week is led by the admission that his comprehension of quantum computing is fleeting at best. Sometimes he gets it, and other times he doesn’t. Something that might help him get over the quantum hump is a series of comic books published by the NSF’s EPiQC that cover quantum computing history and how it works.

Be sure to download this episode of Radio Free HPC, you could be our 16th listener!

Download the MP3 * Subscribe on iTunes * RSS Feed

Sign up for our insideHPC Newsletter