Metaflow: Netflix Machine Learning Platform with Savin Goyal
53 min

Netflix runs all of its infrastructure on Amazon Web Services. This includes business logic, data infrastructure, and machine learning. By tightly coupling itself to AWS, Netflix has been able to move faster and have strong defaults about engineering decisions. And today, AWS has such an expanse of services that it can be used as a platform to build custom tools.

Metaflow is an open source machine learning platform built on top of AWS that allows engineers at Netflix to build directed acyclic graphs for training models. These DAGs get deployed to AWS as Step Functions, a serverless orchestration platform.

Savin Goyal is a machine learning engineer with Netflix, and he joins the show to talk about the machine learning challenges within Netflix, and his experience working on Metaflow. We also talk about DAG systems such as AWS Step Functions and Airflow.

Sponsorship inquiries:

The post Metaflow: Netflix Machine Learning Platform with Savin Goyal appeared first on Software Engineering Daily.

Google Cloud Platform Podcast
Google Cloud Platform Podcast
Google Cloud Platform
2020 Year End Wrap Up
This week, four of the podcast’s greatest hosts come together to celebrate all of the fun and informative episodes we’ve been privileged to do this year! Join Mark Mirchandani, Jon Foust, Priyanka Vergadia, and Brian Dorsey as we talk about our favorite guests and shows, some cool things that happened this year, and what we’re looking forward to in 2021! Cool things of the week * A Giant List of Google Cloud Resources blog * Google Cloud 4 Words site Our favorite episodes * Jon’s Favorites * GCP Podcast Episode 212: Data Management with Amy Krishnamohan podcast * GCP Podcast Episode 237: NVIDIA with Bryan Catanzaro podcast * Priyanka’s Favorite * GCP Podcast Episode 240: reCAPTCHA Enterprise with Kelly Anderson + Spring ML Potholes with Eric Clark podcast * Mark’s Favorites * GCP Podcast Episode 242: NASA and FDL with James Parr and Madhulika Guhathakurta podcast * GCP Podcast Episode 217: Cost Optimization with Justin Lerma and Pathik Sharma podcast * GCP Podcast Episode 228: Fastly with Tyler McMullen podcast * Brian’s Favorites * GCP Podcast Episode 223: Voice Coding with Emily Shea and Ryan Hileman podcast * GCP Podcast Episode 233: Bare Metal Solution with James Harding and Gurmeet Goindi podcast * GCP Podcast Episode 212: Data Management with Amy Krishnamohan podcast Sound Effects Attribution * “Bad Beep” by RicherLandTV of * “Small Group Laugh 6” by Tim.Kahn of * “It’s Always Night in Space” by JamesSilvera of * “Easy Cheesy” by LoboLoco of
36 min
Towards Data Science
Towards Data Science
The TDS team
60. Rob Miles - Why should I care about AI safety?
Progress in AI capabilities has consistently surprised just about everyone, including the very developers and engineers who build today’s most advanced AI systems. AI can now match or exceed human performance in everything from speech recognition to driving, and one question that’s increasingly on people’s minds is: when will AI systems be better than humans at AI research itself? The short answer, of course, is that no one knows for sure — but some have taken some educated guesses, including Nick Bostrom and Stuart Russell. One common hypothesis is that once an AI systems are better than a human at improving their own performance, we can expect at least some of them to do so. In the process, these self-improving systems would become an even more powerful system that they were previously—and therefore, even more capable of further self-improvement. With each additional self-improvement step, improvements in a system’s performance would compound. Where this all ultimately leads, no one really has a clue, but it’s safe to say that if there’s a good chance that we’re going to be creating systems that are capable of this kind of stunt, we ought to think hard about how we should be building them. This concern among many others has led to the development of the rich field of AI safety, and my guest for this episode, Robert Miles, has been involved in popularizing AI safety research for more than half a decade through two very successful YouTube channels, Robert Miles and Computerphile. He joined me on the podcast to discuss how he’s thinking about AI safety, what AI means for the course of human evolution, and what our biggest challenges will be in taming advanced AI.
45 min
The Cloudcast
The Cloudcast
Cloudcast Media
The State of Kubernetes
Nigel Poulton (@nigelpoulton, Docker & Kubernetes training) talks about the current and future state of Kubernetes, who is using it and why, implementation and operational challenges, and how get started and what tools he recommends *SHOW: *478 *SHOW SPONSOR LINKS:* * Okta - You should not be building your own Auth * Learn how Okta helped Cengage improve student success rates during COVID. * Datadog Security Monitoring Homepage - Modern Monitoring and Analytics * Try Datadog yourself by starting a free, 14-day trial today. Listeners of this podcast will also receive a free Datadog T-shirt. * CloudAcademy - Build hands-on technical skills. Get measurable results.  * Get 50% of the monthly price of CloudAcademy by using code CLOUDCAST *CLOUD NEWS OF THE WEEK *- *PodCTL Podcast is Back (Enterprise Kubernetes) *- *SHOW NOTES:* * *Nigel Poulton Homepage* * *Nigel’s YouTube Channel** * * *Kubernetes Deep Drive (ACloudGuru)* * *Nigel’s training courses on PluralSight* * *Eps. 320 (Nov 2017) - "Docker adds Kubernetes, now what?"* * *Eps.269 (Sept 2016) - "Putting Docker in your brain"* *Topic 1 - *Welcome back to the show. Tell us about some of your most recent projects, and all the places where people can find your training?  *Topic 2 - *Let’s talk about the State of Kubernetes, level of maturity of users/usage. Where do you think we are at the end of 2020?  *Topic 3 - *Who do you find are trying to learn and use Kubernetes, more Infra/Ops, or AppDev, or somewhere in between (DevOps, DevSecOps, SRE, etc.)? *Topic 3a - *Where do you see people succeeding, and where are they still struggling to adjust to the requirements of Kubernetes?  *Topic 4 - *The ecosystem of projects around Kubernetes (immutable Linux, Operators, Service Mesh, Prometheus, OPA, Serverless/Knative, etc.) has gotten very broad. How do you find people are dealing with all these moving pieces?  *Topic 5 - *There are a lot of ways to engage with Kubernetes these days - DIY (OSS), vendor distributions, Kubernetes on your laptop, web IDEs, Hosted Services (simple & complex). What do you recommend to people, or do you see trends about how people best engage with Kubernetes?  *Topic 6 - *Do you think it’s OK that Kubernetes doesn’t have a “developer model”, but rather many different options? Do you think this evolves in 2021?   *FEEDBACK?* * Email: show at thecloudcast dot net * Twitter: @thecloudcastnet
34 min
Kubernetes Podcast from Google
Kubernetes Podcast from Google
Adam Glick and Craig Box
KubeCon NA 2020, with Stephen Augustus
Join us for all the news from KubeCon NA 2020, and a conversation with conference co-chair Stephen Augustus. Stephen is a Senior Open Source Engineer on the VMware Tanzu team, a chair of Kubernetes’ SIG Release, and a leader in many other parts of the project, past and present. Do you have something cool to share? Some questions? Let us know: * web: * mail: * twitter: @kubernetespod Chatter of the week * The kākāpō wins Bird of the Year * We’re off for 2 weeks. See you on December 8! News of the week * Cisco acquires Banzai Cloud * CNCF announces Cloud Native Survey 2020 results * Red Hat: New edge features, industrial AI/ML blueprint and AWS launch * CNCF End User Tech Radar for storage * New End User benefits * Envoy Mobile joins the CNCF * New sandbox projects * cert-manager * cdk8s * Kyverno * OpenKruise * Pravega * SchemaHero * Tinkerbell * k8ssandra from Datastax * Episode 98 with Sam Ramji * k0s from Mirantis * announces Gloo Mesh Enterprise and rebrands products * Episode 55, with Idit Levine * Pinniped * Shipa launches Ketch * Kinvolk launches Headlamp * The SPIFFE book “Solving The Bottom Turtle” * Episode 45, with Andrew Jessup * Anthos Developer Sandbox * GKE ingress features * Ambassador Labs takes in $18m and launches v1.9 * Tanzu SQL: Postgres on Kubernetes * Lightning round: * Accurics extends Terrascan * AWS adds containers to Lightsail * Arrikto takes $10m in funding * Brobridge releases Gravity * CircleCI runner is GA * Cloud66 for agencies and multiple database support * Cloudflare Origin CA cert-manager plugin * Cloudical Vanillastack * Cloudify version 5.1 * Codefresh launches GitOps 2.0 features * Commvault backup-as-a-service * Diamanti Spektra 3.1 and customer portal * Dynatrace PurePath 4 * Elastisys Compliant Kubernetes * The Fairwinds Kubernetes Maturity Model * Garden takes “seed” funding * Gremlin adds soundproofing * Humio Operator * Instana adds observability tools on Kubernetes * Intuit runs TurboTax on Kubernetes * Kioxia announces a new storage offering * Kubecost adds features for monitoring outside a cluster * KubeMQ adds automatic network creation * Kubermatic updates KubeOne to v1.1 * Kubernative SINA * Kublr 1.19 * Lablup announced 20.09 RC * Magalix launches KubeAdvisor 2.0 * Mayadata launches Kubera Propel and Kubera Chaos * Mirantis adds extensions to Lens * Puppet Labs adds Relay to Puppet Enterprise * Reblaze announces Curiefense to add WAF to Envoy * Replicates wants to help you Troubleshoot * Styra adds new editions to DAS * Sysdig introduces Kubernetes-native network security (ZTNSK) and partners with IBM Cloud * TrilioVault for Kubernetes v2.0 * Zerto for Kubernetes * Google Open Source Live Kubernetes Links from the interview * KubeCon NA 2020 * Episode 117, with Constance Caramanolis * CNCF Twitch * SIG Friday: ping Stephen for the current link * Slack * CNCF Slack * Kubernetes Slack * Hallway Track * Kubernetes Podcast chat * CoreOS * CoreOS Tectonic * CoreOS acquired by Red Hat * Tectonic on Azure * SIG Azure * SIG Release * SIG PM (retired) * Kubernetes Enhancement Process * Receipts process KEP * Sidecar containers - KEP closed! * Production readiness review * Episode 10, with Josh Berkus and Tim Pepper * Release managers * Black Lives Matter announcement banner * Better announcements * Kubernetes Naming working group * Inclusive Naming project * Dan Kohn memorial * Stephen Augustus on Twitter and on the web
53 min
More episodes
Clear search
Close search
Google apps
Main menu