Metaflow: Netflix Machine Learning Platform with Savin Goyal
53 min

Netflix runs all of its infrastructure on Amazon Web Services. This includes business logic, data infrastructure, and machine learning. By tightly coupling itself to AWS, Netflix has been able to move faster and have strong defaults about engineering decisions. And today, AWS has such an expanse of services that it can be used as a platform to build custom tools.

Metaflow is an open source machine learning platform built on top of AWS that allows engineers at Netflix to build directed acyclic graphs for training models. These DAGs get deployed to AWS as Step Functions, a serverless orchestration platform.

Savin Goyal is a machine learning engineer with Netflix, and he joins the show to talk about the machine learning challenges within Netflix, and his experience working on Metaflow. We also talk about DAG systems such as AWS Step Functions and Airflow.

Sponsorship inquiries:

The post Metaflow: Netflix Machine Learning Platform with Savin Goyal appeared first on Software Engineering Daily.

Google Cloud Platform Podcast
Google Cloud Platform Podcast
Google Cloud Platform
2020 Year End Wrap Up
This week, four of the podcast’s greatest hosts come together to celebrate all of the fun and informative episodes we’ve been privileged to do this year! Join Mark Mirchandani, Jon Foust, Priyanka Vergadia, and Brian Dorsey as we talk about our favorite guests and shows, some cool things that happened this year, and what we’re looking forward to in 2021! Cool things of the week * A Giant List of Google Cloud Resources blog * Google Cloud 4 Words site Our favorite episodes * Jon’s Favorites * GCP Podcast Episode 212: Data Management with Amy Krishnamohan podcast * GCP Podcast Episode 237: NVIDIA with Bryan Catanzaro podcast * Priyanka’s Favorite * GCP Podcast Episode 240: reCAPTCHA Enterprise with Kelly Anderson + Spring ML Potholes with Eric Clark podcast * Mark’s Favorites * GCP Podcast Episode 242: NASA and FDL with James Parr and Madhulika Guhathakurta podcast * GCP Podcast Episode 217: Cost Optimization with Justin Lerma and Pathik Sharma podcast * GCP Podcast Episode 228: Fastly with Tyler McMullen podcast * Brian’s Favorites * GCP Podcast Episode 223: Voice Coding with Emily Shea and Ryan Hileman podcast * GCP Podcast Episode 233: Bare Metal Solution with James Harding and Gurmeet Goindi podcast * GCP Podcast Episode 212: Data Management with Amy Krishnamohan podcast Sound Effects Attribution * “Bad Beep” by RicherLandTV of * “Small Group Laugh 6” by Tim.Kahn of * “It’s Always Night in Space” by JamesSilvera of * “Easy Cheesy” by LoboLoco of
36 min
Towards Data Science
Towards Data Science
The TDS team
60. Rob Miles - Why should I care about AI safety?
Progress in AI capabilities has consistently surprised just about everyone, including the very developers and engineers who build today’s most advanced AI systems. AI can now match or exceed human performance in everything from speech recognition to driving, and one question that’s increasingly on people’s minds is: when will AI systems be better than humans at AI research itself? The short answer, of course, is that no one knows for sure — but some have taken some educated guesses, including Nick Bostrom and Stuart Russell. One common hypothesis is that once an AI systems are better than a human at improving their own performance, we can expect at least some of them to do so. In the process, these self-improving systems would become an even more powerful system that they were previously—and therefore, even more capable of further self-improvement. With each additional self-improvement step, improvements in a system’s performance would compound. Where this all ultimately leads, no one really has a clue, but it’s safe to say that if there’s a good chance that we’re going to be creating systems that are capable of this kind of stunt, we ought to think hard about how we should be building them. This concern among many others has led to the development of the rich field of AI safety, and my guest for this episode, Robert Miles, has been involved in popularizing AI safety research for more than half a decade through two very successful YouTube channels, Robert Miles and Computerphile. He joined me on the podcast to discuss how he’s thinking about AI safety, what AI means for the course of human evolution, and what our biggest challenges will be in taming advanced AI.
45 min
Kubernetes Podcast from Google
Kubernetes Podcast from Google
Adam Glick and Craig Box
KubeCon NA 2020, with Stephen Augustus
Join us for all the news from KubeCon NA 2020, and a conversation with conference co-chair Stephen Augustus. Stephen is a Senior Open Source Engineer on the VMware Tanzu team, a chair of Kubernetes’ SIG Release, and a leader in many other parts of the project, past and present. Do you have something cool to share? Some questions? Let us know: * web: * mail: * twitter: @kubernetespod Chatter of the week * The kākāpō wins Bird of the Year * We’re off for 2 weeks. See you on December 8! News of the week * Cisco acquires Banzai Cloud * CNCF announces Cloud Native Survey 2020 results * Red Hat: New edge features, industrial AI/ML blueprint and AWS launch * CNCF End User Tech Radar for storage * New End User benefits * Envoy Mobile joins the CNCF * New sandbox projects * cert-manager * cdk8s * Kyverno * OpenKruise * Pravega * SchemaHero * Tinkerbell * k8ssandra from Datastax * Episode 98 with Sam Ramji * k0s from Mirantis * announces Gloo Mesh Enterprise and rebrands products * Episode 55, with Idit Levine * Pinniped * Shipa launches Ketch * Kinvolk launches Headlamp * The SPIFFE book “Solving The Bottom Turtle” * Episode 45, with Andrew Jessup * Anthos Developer Sandbox * GKE ingress features * Ambassador Labs takes in $18m and launches v1.9 * Tanzu SQL: Postgres on Kubernetes * Lightning round: * Accurics extends Terrascan * AWS adds containers to Lightsail * Arrikto takes $10m in funding * Brobridge releases Gravity * CircleCI runner is GA * Cloud66 for agencies and multiple database support * Cloudflare Origin CA cert-manager plugin * Cloudical Vanillastack * Cloudify version 5.1 * Codefresh launches GitOps 2.0 features * Commvault backup-as-a-service * Diamanti Spektra 3.1 and customer portal * Dynatrace PurePath 4 * Elastisys Compliant Kubernetes * The Fairwinds Kubernetes Maturity Model * Garden takes “seed” funding * Gremlin adds soundproofing * Humio Operator * Instana adds observability tools on Kubernetes * Intuit runs TurboTax on Kubernetes * Kioxia announces a new storage offering * Kubecost adds features for monitoring outside a cluster * KubeMQ adds automatic network creation * Kubermatic updates KubeOne to v1.1 * Kubernative SINA * Kublr 1.19 * Lablup announced 20.09 RC * Magalix launches KubeAdvisor 2.0 * Mayadata launches Kubera Propel and Kubera Chaos * Mirantis adds extensions to Lens * Puppet Labs adds Relay to Puppet Enterprise * Reblaze announces Curiefense to add WAF to Envoy * Replicates wants to help you Troubleshoot * Styra adds new editions to DAS * Sysdig introduces Kubernetes-native network security (ZTNSK) and partners with IBM Cloud * TrilioVault for Kubernetes v2.0 * Zerto for Kubernetes * Google Open Source Live Kubernetes Links from the interview * KubeCon NA 2020 * Episode 117, with Constance Caramanolis * CNCF Twitch * SIG Friday: ping Stephen for the current link * Slack * CNCF Slack * Kubernetes Slack * Hallway Track * Kubernetes Podcast chat * CoreOS * CoreOS Tectonic * CoreOS acquired by Red Hat * Tectonic on Azure * SIG Azure * SIG Release * SIG PM (retired) * Kubernetes Enhancement Process * Receipts process KEP * Sidecar containers - KEP closed! * Production readiness review * Episode 10, with Josh Berkus and Tim Pepper * Release managers * Black Lives Matter announcement banner * Better announcements * Kubernetes Naming working group * Inclusive Naming project * Dan Kohn memorial * Stephen Augustus on Twitter and on the web
53 min
Streaming Audio: A Confluent podcast about Apache Kafka
Streaming Audio: A Confluent podcast about Apache Kafka
Confluent, original creators of Apache Kafka®
Tales From The Frontline of Apache Kafka Devops ft. Jason Bell
Jason Bell (Apache Kafka® DevOps Engineer,, and Author of “Machine Learning: Hands-On for Developers and Technical Professionals” ) delves into his 32-year journey as a DevOps engineer and how he discovered Apache Kafka. He began his voyage in hardware technology before switching over to software development. From there, he got involved in event streaming in the early 2000s where his love for Kafka started. His first Kafka project involved monitoring Kafka clusters for flight search data, and he's been making magic ever since! Jason first learned about the power of the event streaming during Michael Noll’s talk on the streaming API in 2015. It turned out that Michael had written off 80% of Jason’s streaming API jobs with a single talk.  As a Kafka DevOps engineer today, Jason works with on-prem clusters and faces challenges like instant replicas going down and bringing other developers who are new to Kafka up to speed so that they can eventually adopt it and begin building out APIs for Kafka. He shares some tips that have helped him overcome these challenges and bring success to the team. EPISODE LINKS * Machine Learning: Hands-On for Developers and Technical Professionals by Jason Bell  * Join the Confluent Community Slack * Learn more with Kafka tutorials, resources, and guides at Confluent Developer * Live demo: Kafka streaming in 10 minutes on Confluent Cloud * Use *60PDCAST *to get an additional $60 of free Confluent Cloud usage (details)
1 hr
Soft Skills Engineering
Soft Skills Engineering
Jamison Dance and Dave Smith
Episode 238: Naughty team and quitting after 2 weeks
In this episode, Dave and Jamison answer these questions: Questions * A few years ago, my current company did a big no-no which turned into a scandal that made national headlines. When I was considering joining, I said it was important for me to feel ethically aligned with my work, and asked about how things had changed since The Incident. They told me they stopped doing bad things, and I accepted the offer. Well, during my time at the company, it has slowly been dawning on me that my team is THE TEAM in question. I finally gathered the courage to ask a coworker, and he confirmed that this was true, and that there’s more designs coming down the pipeline that he and other devs are uncomfortable building. He brought it up with our manager and he was basically told “business is business”. As devs, we don’t make the decisions. And our golden handcuffs are really shiny. Should I leave, stay and try to influence change from the inside, or stay and maybe be a whistleblower one day if need be? * I think I made a horrible mistake. I gave up an undesirable job for a fairly large tech company, and joined a Drupal agency. These two weeks have been the longest year of my life. I haven’t written one line of code, and the Drupal admin is incomprehensible. Since it’s only been a (relatively) short time here, how do I get back in the job market without looking like a chump? Do I remove it from my resume? Do I own it like a hideous tattoo? What do I tell hiring managers; whether its a gap in my resume, or that I want to leave after only 2 weeks? Any and all help is appreciated. Thank you!
32 min
More episodes
Clear search
Close search
Google apps
Main menu