Frontiers of Performance with Daniel Lemire
Play • 47 min

Did you ever meet somebody who seemed a little bit different than the rest of the world? Maybe they question things that others wouldn’t question or said things that others would never say.

Daniel is a world-renowned expert on software performance, and one of the most popular open source developers, if you measure by get up followers. Today, he’s gonna share his story. It involves time at a research lab, teaching students in a new way. It will also involve upending people’s assumptions about IO performance.  Elon Musk And Julia Roberts will come up a little bit more than you might expect.

Episode Page

Episode Transcript

Links:

Daniel's Blog

Daniel's Github

Parsing JSON Really Quickly: Lessons Learned

Changelog
Changelog
Practical AI 119: Accelerating ML innovation at MLCommons
MLCommons launched in December 2020 as an open engineering consortium that seeks to accelerate machine learning innovation and broaden access to this critical technology for the public good. David Kantor, the executive director of MLCommons, joins us to discuss the launch and the ambitions of the organization. In particular we discuss the three pillars of the organization: Benchmarks and Metrics (e.g. MLPerf), Datasets and Models (e.g. People’s Speech), and Best Practices (e.g. MLCube). Discuss on Changelog News Join Changelog++ to support our work, get closer to the metal, and make the ads disappear! Sponsors * Code-ish by Heroku – A podcast from the team at Heroku, exploring code, technology, tools, tips, and the life of the developer. Check out episode 98 and episode 99 for insights on the ethical and technical sides of deep fakes. Subscribe on Apple Podcasts and Spotify. * Changelog++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this! * Knowable – Learn from the world’s best minds, anytime, anywhere, and at your own pace through audio. Get unlimited access to every Knowable audio course right now. Click here to check it out and use code CHANGELOG for 20% off! Featuring * David Kanter – Twitter, GitHub * Chris Benson – Twitter, GitHub, LinkedIn, Website * Daniel Whitenack – Twitter, GitHub, Website Notes and Links * Learn more about People’s Speech * Get involved with the People’s Speech project * MLCube GitHub, including several different examples * MLCube Mailing list * MLPerf Training Benchmarks * MLPerf Training HPC Benchmarks * MLPerf Inference Datacenter Benchmarks * MLPerf Inference Edge Benchmarks * MLPerf Inference Mobile Benchmarks * MLCommons on Twitter
Play
Python Bytes
Python Bytes
Michael Kennedy and Brian Okken
#216 Container: Sort thyself!
Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Jousef Murad, Engineered Mind podcast (audio, video) Watch on YouTube Brian #1: pip search. Just don’t. * pip search [query] is supposed to “Search for PyPI packages whose name or summary contains [query]” * The search feature looks like it’s going to be removed and the PyPI api for it removed. * Alternative, and better approach, just manually look at pypi.org and search for stuff. * Right now it does this: $ pip search pytest ERROR: Exception: Traceback (most recent call last): ... [longish traceback ommited] --- xmlrpc.client.Fault: [Fault -32500: "RuntimeError: PyPI's XMLRPC API has been temporarily disabled due to unmanageable load and will be deprecated in the near future. See https://status.python.org/ for more information."] * The Python Infrastructure status page says, as of Jan 12: “Update - The XMLRPC Search endpoint remains disabled due to ongoing request volume. As of this update, there has been no reduction in inbound traffic to the endpoint from abusive IPs and we are unable to re-enable the endpoint, as it would immediately cause PyPI service to degrade again.” * This started becoming a problem in mid December. * The endpoint was just never architected to handle the scale it’s getting now. * There’s a current issue “Remove the pip search command”, open on pip. * The commend thread is locked now, but you can read some of the history. * I personally don’t understand the need to hammer search with a CI system or other. * Probably should be using a local cache or local pypi mirror for an active/aggressive CI system. * If you have scripts or jobs that run pip search , it ain’t gonna work, so probably best to remove that. Michael #2: QPython - Scripting for Android with Python * Python REPL on Android - interesting * Scripting Android tasks with Python - more interesting * Free, open source app that is ad supported. * Some people have commented that their phone is their only “computer” * With SL4A features, you can use Python programming to control Android work: * Android Apps API, such as: Application, Activity, Intent & startActivity, SendBroadcast, PackageVersion, System, Toast, Notify, Settings, Preferences, GUI * Android Resources Manager, such as: Contact, Location, Phone, Sms, ToneGenerator, WakeLock, WifiLock, Clipboard, NetworkStatus, MediaPlayer * Third App Integrations, such as: Barcode, Browser, SpeechRecongition, SendEmail, TextToSpeech * Hardwared Manager: Carmer, Sensor, Ringer & Media Volume, Screen Brightness, Battery, Bluetooth, SignalStrength, WebCam, Vibrate, NFC, USB Jousef #3: Thesis: Deep Learning assistant for designers/engineers * PyTorch (3D) / TensorFlow * The thesis: what is it actually about & goal of the thesis * Libraries mainly used: numpy, pandas * (Reinforcement Learning & GANs) Brian #4: sortedcontainers * Thanks to Fanchen Bao for the topic suggestion. * Pure-Python, as fast as C-extensions, sorted collections library. >>> from sortedcontainers import SortedList >>> sl = SortedList(['e', 'a', 'c', 'd', 'b']) >>> sl SortedList(['a', 'b', 'c', 'd', 'e']) >>> sl *= 10_000_000 >>> sl.count('c') 10000000 >>> sl[-3:] ['e', 'e', 'e'] >>> from sortedcontainers import SortedDict >>> sd = SortedDict({'c': 3, 'a': 1, 'b': 2}) >>> sd SortedDict({'a': 1, 'b': 2, 'c': 3}) >>> sd.popitem(index=-1) ('c', 3) >>> from sortedcontainers import SortedSet >>> ss = SortedSet('abracadabra') >>> ss SortedSet(['a', 'b', 'c', 'd', 'r']) >>> ss.bisect_left('c') 2 * “All of the operations shown above run in faster than linear time.” * Types: * SortedList * SortedKeyList (like SortedList, but you pass in a key function, similar to key in Pythons sorted function.) * SortedDict * SortedSet * Great documentation and tons of performance metrics in the docs. Michael #5: Łukasz Langa Typed Twitter Thread * Let’s riff on typing for a bit. * Here is my philosophy: If I have to type more than three characters to complete a symbol in my editor, something is wrong. * e.g. to go from email_service. → email_service.send_account_email() I should only need to type .sae then tab/enter. These types of things are vastly better because of type hints. * Python type hints are more malleable than even TypeScript. * Lukasz is addressing this comment: Controversial take: Types in a Python code-base are a net negative. * Points * put enough annotations and tooling connects the dots, making plenty of errors evident. * The most common to me at least is when a None creeps in. * The second bug often caught by type checkers is on the "return" boundary: one of your code paths forgets a return. * squiggly lines in your editor * Microsoft is now developing powerful type checking and code completion for Python in VSCode. This effort employs a member of the Python Steering Council, and possibly also the creator of Python himself soon. You think they would settle for "illusion of productivity"? Jousef #6: * Point Cloud operations → open3d Extras: Michael: * via Francisco Giordano Silva: On Brian's ref to using numpy all for array element-wise comparison, also please check out numpy.allclose method. Allows you to compare two arrays based on a given tolerance. Brian: * Just this: 2021 is exhausting so far. * Test & Code has shifted to every other week to allow time for other projects I’m working on. * This is probably a short term change. But I don’t know for how long. It’s definitely not going away though. Just slowing down a bit. Jousef: Engineered Mind podcast
36 min
Software Defined Talk
Software Defined Talk
Software Defined Talk LLC
Episode 281: That’s a thing, I don’t need to read about it
This week we discuss VMware CEO Pat Gelsinger jumping to Intel and what is going on with DevSecOps. Plus, lots advice on picking movies both you and your partner will enjoy. Rundown VMware CEO → Intel What’s the latest Solarwinds hack news? Coté is figuring out “DevSecOps” - or is it “DevOpsSec”? Relevant to your interests M&A Red Hat to Acquire Kubernetes-Native Security Leader StackRox (https://www.redhat.com/en/about/press-releases/red-hat-acquire-kubernetes-native-security-leader-stackrox) F5 to acquire @Volterra_ (https://twitter.com/f5/status/1347291942363811841?s=21) VMware/Intel VMware has the strategy and culture to thrive after CEO Pat Gelsinger's exit to Intel (https://www.theregister.com/2021/01/14/pat_gelsinger_vmware_legacy/) VMware Names Zane Rowe As Interim CEO (https://www.crn.com/news/virtualization/vmware-names-zane-rowe-as-interim-ceo) Prodigal Son Gelsinger Returns As Intel CEO (https://go.forrester.com/blogs/prodigal-son-gelsinger-returns-as-intel-ceo/) Bitcoin and Blockchain Is blockchain coming to your bank? (https://thehustle.co/01082021-blockchain-banks/) 85% of Italian Banks Are Exchanging Interbank Transfer Data on Corda - CoinDesk (https://www.coindesk.com/85-of-italian-banks-are-exchanging-interbank-transfer-data-on-corda) Lost Passwords Lock Millionaires Out of Their Bitcoin Fortunes (https://www.nytimes.com/2021/01/12/technology/bitcoin-passwords-wallets-fortunes.html) Don’t Forget Your Bitcoins (https://www.bloomberg.com/opinion/articles/2021-01-12/don-t-forget-your-bitcoins) RISC-V BeagleBoard BeagleV (https://beagleboard.org/beaglev) Raspberry PI Why We Love the Raspberry Pi (https://www.nytimes.com/wirecutter/reviews/raspberry-pi/) Buy a Raspberry Pi Zero W – Raspberry Pi (https://www.raspberrypi.org/products/raspberry-pi-zero-w/) Solarwinds Details on SolarWinds Hack (https://twitter.com/briankrebs/status/1348828797966147584?s=21) SolarWinds hires former Trump cyber security chief Chris Krebs (https://www.ft.com/content/df641e33-9150-4846-b4f7-db4e3175d290) NYT JetBrains story, I'm calling it - the story was irresponsibly released. (https://twitter.com/MalwareJake/status/1347650824416227331) Deplatforming Parler accuses Amazon of breaking antitrust law in suspending hosting services. (https://www.nytimes.com/2021/01/11/business/parler-amazon.html) Parler Finds Refuge With the Far-Right's Favorite Webhost (https://www.vice.com/en/article/4ad7dp/parler-finds-refuge-with-the-far-rights-favorite-webhost) Parler loses data (https://twitter.com/salmeron_manny/status/1348604719934173185) Parler’s amateur coding could come back to haunt Capitol Hill rioters (https://arstechnica.com/information-technology/2021/01/parlers-amateur-coding-could-come-back-to-haunt-capitol-hill-rioters/) The balkanization of the cloud is bad for everyone (https://www.technologyreview.com/2020/12/17/1014967/balkanization-cloud-computing-bad-everyone/) Decentralization Is A Necessity Now (https://pomp.substack.com/p/decentralization-is-a-necessity-now) It happened. Twitter shut down @realDonaldTrump for good (https://thehustle.co/01112021-Twitter-Trump/) State of the World 2021 (https://people.well.com/conf/inkwell.vue/topics/510/State-of-the-World-2021-page01.html) Goodreads plans to retire API access, disables existing API keys | Joe's website (https://joealcorn.co.uk/blog/2020/goodreads-retiring-API) ‘Your Cock Is Mine Now:’ Hacker Locks Internet-Connected Chastity Cage, Demands Ransom (https://www.vice.com/en/article/m7apnn/your-cock-is-mine-now-hacker-locks-internet-connected-chastity-cage-demands-ransom) Video games have replaced music as the most important aspect of youth culture | Sean Monahan (https://www.theguardian.com/commentisfree/2021/jan/11/video-games-music-youth-culture) Intel CEO Bob Swan to be replaced with VMWare's Pat Gelsinger (https://finance.yahoo.com/news/intel-ceo-bob-swan-reportedly-set-to-depart-in-february-to-be-replaced-with-vm-wares-pat-gelsinger-142520631.html?guccounter=1) WhatsApp gives users an ultimatum: Share data with Facebook or stop using the app (https://arstechnica.com/tech-policy/2021/01/whatsapp-users-must-share-their-data-with-facebook-or-stop-using-the-app/) First Oracle said it powered Zoom. Then AWS claimed it. Now Zoom says it uses co-located kit (https://www.theregister.com/2021/01/13/zoom_prospectus_reveals_colo_infrastructure/) Poland plans to make censoring of social media accounts illegal (https://www.theguardian.com/world/2021/jan/14/poland-plans-to-make-censoring-of-social-media-accounts) Dropbox to cut workforce by 11% (https://www.axios.com/dropbox-to-cut-workforce-by-11-75f8c050-29bb-46ea-a7dc-d9fa972dcee2.html?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_axioslogin&stream=top) Nonsense I'm glad you have passed the certification! I just don't want to see it. (https://github.com/antonbabenko/you-have-passed-the-certification) Coté in TikTok (https://www.tiktok.com/@drunkandretired/video/6917223569395174658?lang=en). Sponsors strongDM — Manage and audit remote access to infrastructure. Start your free 14-day trial today at: strongdm.com/SDT (http://strongdm.com/SDT) Now Hiring Jordi wants you to work at Tricentis (https://www.tricentis.com/company/careers/all/) Michael wants to be a Solution Architect at Cloudbees (https://boards.greenhouse.io/cloudbees/jobs/2582391) More info in SDT Slack (https://www.softwaredefinedtalk.com/slack) see channel #jobs Conferences Call for Papers (https://sessionize.com/devopsdays-texas-2021/) ends on Jan. 31st for DevOpsDay Texas on March 2nd. (https://devopsdays.org/events/2021-texas/welcome/) SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Send your postal address to stickers@softwaredefinedtalk.com (mailto:stickers@softwaredefinedtalk.com) and we will send you free laptop stickers! Follow us on Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/) and LinkedIn (https://www.linkedin.com/company/software-defined-talk/). Brandon built the Quick Concall iPhone App (https://itunes.apple.com/us/app/quick-concall/id1399948033?mt=8) and he wants you to buy it for $0.99. Use the code SDT to get $20 off Coté’s book, (https://leanpub.com/digitalwtf/c/sdt) Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Matt: Apple TV’s Ted Lasso (https://tv.apple.com/au/show/ted-lasso/umc.cmc.vtoh0mn0xn7t3c643xqonfzy) Brandon: Possessor (https://www.rottentomatoes.com/m/possessor_uncut). Coté: Descript (https://www.descript.com), so far. Bear app (https://bear.app). Don’t spike your mind during family time with inserting work. Photo Credit (https://unsplash.com/photos/ny-lHmsHYHk) Photo Credit (https://unsplash.com/photos/eM6WUs4nKMY)
1 hr 2 min
More episodes
Search
Clear search
Close search
Google apps
Main menu