Slack Data Platform with Josh Wills Holiday Repeat
Play • 1 hr 18 min

Originally published January 10, 2020

Slack is a messaging platform for organizations. Since its creation in 2013, Slack has quickly become a core piece of technology used by a wide variety of technology companies, groups, and small teams. 

The messages that are sent on Slack are generated at a very high volume, and are extremely sensitive. These messages must be stored on Slack’s servers in a way that does not risk a message from one company accidentally being accessible to another company. The messages must be highly available, and they also must be indexed for search.

When Slack was scaling, the company started to encounter limitations in its data infrastructure that the company was unsure how to solve. During this time, Josh Wills was the director of data engineering at Slack, and he joins the show to retell the history of his time at Slack, and why the problem of searching messages was so hard. 

Josh also provides a great deal of industry context around how engineers from Facebook and Google differ from one another. When Slack was starting to become popular, the company quickly began to attract engineers from both of those companies. Facebook and Google have distinct solutions for how they have tackled the problems of data engineering.

The post Slack Data Platform with Josh Wills Holiday Repeat appeared first on Software Engineering Daily.

Data Futurology - Leadership And Strategy in Artificial Intelligence, Machine Learning, Data Science
Data Futurology - Leadership And Strategy in Artificial Intelligence, Machine Learning, Data Science
Felipe Flores
#139 Causal Inference with Carlos Avello - Marketing Science Lead
We have Carlos Avello, Marketing Science Lead at Amazon (Director of Marketing Analytics at eBay at the time of recording) joining us for an insightful chat on Causal Inference and how it can help businesses understand causes and impacts for better decision making. Even before ecommerce started, Carlos was already working within an extremely data-driven business model. He started his journey in direct response marketing, tracking data from coupons and later analyzing it. In his own words, he explains causal inference is based on evidence you collect of the impact a certain marketing effort had on your overall sales. Experimentation is the core of causal inference, and specifically A/B tests are used very often. Quotes: * "Causal inference is based on evidence you collect of the impact a certain marketing effort had on your overall sales." * "Experimentation is at the core of it. You cannot get into causal inference if you do not compare two different populations with different treatments." * "Marketing is the product of a partnership between the advertiser and the advertising platform. Two different companies and there are things that we are ready to share and some others that are tricky to share, like customer data. If you want to setup a test where you are suppressing treatment to some people and allowing others to be exposed to the treatment and then compare the final results on sales you need to find a solution were you are saving enough information so the advertiser knows who has been exposed and who has not and later they can compare the sales." Thanks to our sponsor: Talent Insights Read the full episode summary here: Ep #139 Enjoy the show! --- Send in a voice message:
57 min
Streaming Audio: A Confluent podcast about Apache Kafka
Streaming Audio: A Confluent podcast about Apache Kafka
Confluent, original creators of Apache Kafka®
Scaling Developer Productivity with Apache Kafka ft. Mohinish Shaikh
Confluent Cloud and Confluent Platform run efficiently largely because of the dedication of the Developer Productivity (DevProd) team, formerly known as the Tools team. Mohinish Shaikh (Software Engineer, Confluent) talks to Tim Berglund about how his team builds the software tooling and automation for the entire event streaming platform and ensures seamless delivery of several engineering processes across engineering and the rest of the org. With the right tools and the right data, developer productivity can understand the overall effectiveness of a development team and their ability to produce results. The DevProd team helps engineering teams at Confluent ship code from commit to end customers actively using Apache Kafka®. This team proficiently understands a wide scope of polyglot applications and also the complexities of using a diverse technology stack on a regular basis to help solve business-critical problems for the engineering org.  The team actively measures how each system interacts with one another and what programs are needed to properly run the code in various environments to help with the release of reliable artifacts for Confluent Cloud and Confluent Platform. An in-depth understanding of the entire framework and development workflow is essential for organizations to deliver software reliably, on time, and within their cost budget. The DevProd team provides that second line of defense and reliability before the code is released to end customers. As the need for compliance increases and the event streaming platform continues to evolve, the DevProd team is in place to make sure that all of the final touches are completed.  EPISODE LINKS * Leveraging Microservices and Apache Kafka to Scale Developer Productivity * Join the Confluent Community Slack * Learn more with Kafka tutorials, resources, and guides at Confluent Developer * Live demo: Kafka streaming in 10 minutes on Confluent Cloud * Use *60PDCAST* to get an additional $60 of free Confluent Cloud usage (details)
34 min
Azure DevOps Podcast
Azure DevOps Podcast
Jeffrey Palermo
Charles Flatt on Learning as a Developer - Episode 124
Charles Flatt is joining the podcast today! He has been a software developer since 1994 and has helped over a dozen organizations succeed on over fifty projects, both small and large. Charles has an unusual breadth of business and personal experience from foodservice and retail to music, business management, hardware installation, and of course, software development. In this episode, Charles talks about his learning as a developer and some of his successes, big lessons, and key takeaways from the course of his career. He shares actionable advice for developers, teams, and organizations on how to improve; his favorite resources and books for further learning; the metrics that matter the most; and what he sees as being the key components of what makes a DevOps organization successful. Topics of Discussion: [:38] Be sure to visit AzureDevOps.Show for past episodes and show notes. [1:00] About The Azure DevOps Podcast, Clear Measure, and Jeffrey’s offer to speak at virtual user groups. [1:24] Clear Measure is hiring! Be sure to check out the link in the show notes. [1:34] About today’s guest, Charles Flatt! [1:57] Jeffrey welcomes Charles to the podcast. [2:34] Charles shares his career journey before software development and how he began his career in software. [6:49] Charles speaks about where he has worked and what he has been working on in the last decade. [11:48] Charles shares some of the big lessons and key takeaways from the course of working on over fifty projects in software development. [18:21] Charles and Jeffrey discuss their favorite books on DevOps and give their recommendations on what you should be reading as a developer today. [20:50] A word from Azure DevOps Podcast’s sponsor: Clear Measure. [21:22] What Charles sees as needing to change within an organization in order to become more successful. [23:01] Charles gives some actionable advice on how to begin improving as a developer, as a team, and as an organization. [28:06] Charles and Jeffrey discuss the metrics that matter the most. [29:18] Jeffrey and Charles discuss the importance of continuous integration and what it really means to do continuous integration. [32:32] Charles recommends some go-to resources to check out after today’s podcast! [34:33] Jeffrey thanks Charles for joining the podcast! Mentioned in this Episode: Azure DevOps Clear Measure (Sponsor) .NET DevOps for Azure: A Developer's Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! — Click here to download the .NET DevOps for Azure ebook! Jeffrey Palermo’s Youtube Jeffrey Palermo’s Twitter — Follow to stay informed about future events! The Azure DevOps Podcast’s Twitter: @AzureDevOpsShow Charles Flatt’s LinkedIn Azure DevOps Podcast Ep. 33: “Rockford Lhotka on Software Architecture” Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations, by Nicole Forsgren PhD The Phoenix Project (A Novel About IT, DevOps, and Helping Your Business Win), by Gene Kim The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations, by Gene Kim, Patrick Debois, John Willis, and Jez Humble Continuous Integration: Improving Software Quality and Reducing Risk, by Paul M. Duvall, Steve Matyas, and Andrew Glover Good Habits, Bad Habits: The Science of Making Positive Changes That Stick, by Wendy Wood Measure What Matters: How Google, Bono, and the Gates Foundation Rock the World with OKRs, by John Doerr Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.
35 min
Java Pub House
Java Pub House
Freddy Guime & Bob Paulin
Episode 94. Oh, put on your hat Dr. Watson, we are sleuthing this Heap Dump
So it happens. Sometimes a Java program just "dies" with the dreaded Out-of-memory Exception. Sometimes, it leaves behind a "heap dump", or a copy of what the Java program had in memory when just before it throw the Out-of-memory exception. For some devs, a heap dump is stressful, because they think is a black box with only mysteries, but we are here to calm your fears! In this episode we show how Heap dumps are your friends! If you happen to have one, then is almost assured that you can find what caused the out-of-memory, and/or you can "see" what the real values of configuration were. For troubleshooting a production incident, a heap dump becomes invaluable since "nothing" hides from it. Doesn't matter where the code came from, a heap dump will have exactly what each variable was holding and who might be responsible for a memory leak! "Memory leak you say? I thought java didn't have memory leaks!". Well, that's the other part of this episode where we discuss while Java has way less worries than (say c++) on allocating and deallocating memory, you can still create Memory leaks by having strong referenced objects that are never removed (and if that sounded like mumbo jumbo, even a better excuse to listen to this episode) So, we survived 2020. Take a listen to this end-of-the-year episode, and be even more ready for whatever 2021 throws at us! We thank DataDogHQ for sponsoring this podcast episode Don't forget to SUBSCRIBE to our cool NewsCast OffHeap! How to capture a Heap Dump Eclipse Memory Analyzer VisualVM Do you like the episodes? Want more? Help us out! Buy us a beer! And Follow us!
1 hr 6 min
More episodes
Clear search
Close search
Google apps
Main menu