Committing to Cloud Native

Mar 22, 2021

Episode 3: Google Cloud, Hay-doop, Mars Rover, AWS and more with Miles Ward of SADA

Sponsored by Reblaze, creators of Curiefense Panelists Justin Dorfman | Richard Littauer Guest Miles Ward Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the cloud native space. Today, our amazing guest is Miles Ward, who is CTO at SADA, which is Google Cloud’s number one resale and implementation partner. We find what Miles does at SADA, how he helped NASA with the Curiosity Mars Rover landing, a startup he did, his bizarre relationship with Twitter, and he shares some awesome advice for people who are interested in seeing outliers for Cloud Native stuff. Also, find out why Miles tells us how we could be using Kubernetes on the moon one day! Download this episode now to find out so much more! [00:00:59] We start off with some Curiefense updates. [00:02:51] Miles tells us about helping NASA with the Curiosity Mars Rover landing. [00:04:28] We learn what Miles does at SADA and what his experience was with the cloud at AWS. [00:08:44] Miles explains his “bizarre” relationship with Twitter. [00:12:24] Richard wonders why Miles switched from Google to SADA. [00:17:13] Miles tells us why he went to SADA and why wouldn’t Google keep him internal. [00:19:12] Find out what Miles is most excited about and what he’s doing with his extra powers these days. [00:21:23] Richard asks Miles if he has any advice to share for people who are interested in seeing outliers for Cloud Native stuff and how you can plan for that. [00:24:41] We learn if the military uses these data storage sites and diesel generators that Miles was talking about earlier, and if this one of his main clients. He mentions talks with Johnson Space Center using Kubernetes on the moon! [00:27:20] Richard used to work for IPFS and is curious to know what Miles thinks about the decentralized systems and how that’s going to go in the future. [00:30:35] Hear the story how Miles got the gig with Rover on Mars. [00:37:34] Find out where you can follow Miles on the internet. Spotlight [00:38:31] Justin’s spotlight is a CNCF project called Linkerd. [00:39:04] Richard’s spotlight is IPFS. [00:39:18] Miles’s spotlight is NGINX. Quotes [00:03:02] “It was a real trip watching the perseverance landing because it’s like I was in that room, like I remember the last time, let’s see if they don’t screw this up.” [00:07:15] “But Amazon, I spent, you know I built a startup and it was terrible, and we vaporized a bunch of shareholder value. It was outstanding.” [00:07:35] “We read papers about Hadoop and deployed Hadoop at scale, only like four months later to realize we were pronouncing it wrong and we didn’t know what we were doing, and Hadoop is kind of a cool thing now.” [00:08:06] “And I was like you have M1 larges with eight gigs of memory, seven and a half in fact, like not even a whole eight? What did you do with the other half gig?” [00:08:25] “It turns out you can build awesome big stuff with little tiny building blocks.” [00:09:45] “We were the second largest customer of that product behind the CIA.” [00:10:57] “And as a part of that I met Eric Schmidt through the Obama campaign I worked as a part of the 2012 OFA team to help design the data infrastructure for processing all of the campaign ads and analytics and all the rest of the stuff.” [00:11:43] “Google revenue was pretty small and in the five years that I was there we grew to one hundred twelve-fold. “ [00:11:49] “Andy Jassy has this quote at one of the reinvents, I presented I think the first reinvent had sixty sessions and I did seven, so it was a lot of content out there.” [00:11:57] “Jassy had this comment where it was like, “Look, the biggest impediment to AWS growth is the lack of a viable competitor. If only we had a real competition, then companies would have multiple options to select from and they will be able to move to cloud more quickly.” [00:12:31] “So, Google’s incredible. You’re going to build a startup, every one of your little departments and divisions, they’re own little startup, and you now officially have the coolest garage in which to build a startup that there has ever been.” [00:12:43] “They have a fifth of the x86 compute on the planet. They have a network six times the size of this other network you’ve heard of called the internet.” [00:12:52] “They’re putting thirty tons of net new disk in the data centers a day.” [00:13:16] “We were working on the project to work with Twitter to help them understand hey, look, we can handle it.” [00:13:24] “And they go, look, we have 330 petabytes in our Hadoop cluster.” [00:13:46] “Doug Cutting at Yahoo goes, well, this is hot. Let me see if I can sort of reimplement this. He chooses Java because I don’t know, he doesn’t like himself or something.” [00:14:18] “So we run Hadoop tests all the time, and then you can kind of squint back and we’ll performance test logs because they don’t run MapReduce anymore inside Google, but MapReduce is faster than it by a lot.” [00:15:09] “And a petabyte of solid-state disk is a tenth of a percent of the global production of that type of equipment as of that day.” [00:15:45] “You’re like, hold on a minute, ten petabytes of SSD as of 2016 is an absolutely galactic amount of infrastructure.” [00:16:01] “So, I spent a bunch of time with the Pokémon Go folks, they gave us hey, here’s this curve, here’s how much traffic you’re going to use, and here’s this really like worse case right here, and it’s super gnarly, it’s going to be like this and were like okay totally cool, and they used fifty three times that much in a week later.” [00:16:14] “At the peak of that they’re slightly bigger than Gmail.” [00:16:38] “And at the time they’re running on Kubernetes because they’re doing what we told them to do and they’re following best practice.” [00:17:32] “Oh, they love SADA, so they’re stoked.” [00:18:03] “And to be really tactical, like really clear about it, there’s an enormous amount of direct risk when I’m the one who pushes the magic button on your stuff.” [00:18:13] “And if I’m a Google engineer with access to the G3, the core central code base, the multi billion line mono repo that holds the source for surge and the source for MapReduce and every other thing, if I do the wrong commit and push some of the code in my repo into your repo, it’s extra bad.” [00:18:53] “In fact, I think the best of the hyperscalers interaction with open source, but we can really participate in those communities and plug in.” [00:19:19] “One of the big ones that I think is a real opportunity, we spend a lot of cycles thinking about the financial operations. I know it’s not as technical as you may like, but the reality is, totally depend on the financial details.” [00:20:04] “They build the product, they don’t use the product all day, so we use the product all day, and the result of that is I think we’re building a repository of information about its use.” [00:20:26] “It’s a spot where there’s a lot of room because I think your guys’ area in Cloud Native, there’s a bunch of confusion.” [00:20:54] “Like a bunch of noise about this container thing, is that actually a good idea? Does it actually save anybody any money? So that’s a spot where we spend a lot of cycles.” [00:21:34] “Ninety percent of the weird stuff people don’t notice cause they’re not logging.” [00:22:30] “So Google had one where a diesel generator just vaporized a bunch of diesel and just decided to be a fire canon to the sky. It was great!” [00:26:12] “We’re in talking now with the folks at Johnson Space Center about using Kubernetes on the moon, like I joke you not!” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (ht…

41 min