Python Bytes
Jan 13, 2022
#266 Python has a glossary?
playlist_add
See the full show notes for this episode on the website at pythonbytes.fm/266
More episodes from Python Bytes
5 days ago
#283 The sports episode
Watch the live stream: Watch on YouTube
About the show
Sponsored: RedHat: Compiler Podcast
Special guest: Tonya Sims
Michael #1: Pathy: a Path interface for local and cloud bucket storage
* via Spencer
* Pathy is a python package (with type annotations) for working with Cloud Bucket storage providers using a pathlib interface.
* It provides an easy-to-use API bundled with a CLI app for basic file operations between local files and remote buckets.
* It enables a smooth developer experience by letting developers work against the local file system during development and only switch over to live APIs for deployment.
* Also has optional local file caching.
* From Spenser
* The really cool function is "Pathy.fluid" which can take any type of local, GCS, or S3 path string and then just give you back a Path object that you can interact with agnostic of what platform it was. So this has worked amazingly for me in local testing since i can just change the file path from the "s3://bucket/path" that i use in prod to a local "test_dir/path" and it works automatically.
Brian #2: Robyn
* “Robyn is a fast, high-performance Python web framework with a Rust runtime.”
* Hello, Robyn! - intro article
* docs, repo
* Neat things
* doesn’t need WSGI or ASGI
* async
* very Flask-like
* Early, so still needs some TLC
* docs, etc. getting started and demo apps would be good.
Tonya #3: Python package 'nba_api' is a package to access data for NBA.com
* This package is maintained by Swar Patel
* API Client package for NBA.com, more accessible endpoints, and better documentation
* The NBA.com API's are not well documented and change frequently (player traded, injured, retired, points per game, stats, etc)
* The nba_api package has tons of features:
* The nba_api starts with static data on players and teams (Full name, team name, etc). Each player and Team has an id.
* Can get game data from the playergamelog API endpoint
* The package also has many different API endpoints that it can hit by passing in features from the static data to the API endpoints as parameters
Michael #4: Termshot
* From Jay Miller
* Creates screenshots based on terminal command output
* Just run termshot YOUR_CMD
* or termshot --show-cmd -- python program.py
* Even termshot /bin/zsh for full interactive “recording”
* Example I made:
Brian #5: When Python can’t thread: a deep-dive into the GIL’s impact
* Itamar Turner-Trauring
* Building a mental model of the GIL using profiler graphs of simple two thread applications.
* The graphs really help a lot to see when the CPU is active or waiting on each thread.
Tonya #6: Sportsipy: A free sports API written for python
* Free python API that pulls the stats from www.sports-reference.com
* sports-reference.com - great website for getting sports stats for professional sports(NBA, NFL, NHL, MLB, college sports)
* Looks like an HTML website for the 90s - great for scraping (email site owners)
* You can get API queries for every sport (North American sports) like the list of teams for that sport, the date and time of a game, the total number of wins for a team during the season, and many more team-related metrics.
* You can also get stats from players and box scores - so you can build cool stuff around how a team performed during a game or during a season.
Extras
Michael:
* Python 3.11.0 beta 1 is out
* Test with GitHub Actions against Python 3.11
Joke: Finding my family
playlist_add
May 3, 2022
#282 Don't Embarrass Me in Front of The Wizards
Watch the live stream: Watch on YouTube
About the show
Sponsored by us! Support our work through:
* Our courses at Talk Python Training
* Test & Code Podcast
* Patreon Supporters
Brian #1: pyscript
* Python in the browser, from Anaconda. repo here
* Announced at PyConUS
* “During a keynote speech at PyCon US 2022, Anaconda’s CEO Peter Wang unveiled quite a surprising project — PyScript. It is a JavaScript framework that allows users to create Python applications in the browser using a mix of Python and standard HTML. The project’s ultimate goal is to allow a much wider audience (for example, front-end developers) to benefit from the power of Python and its various libraries (statistical, ML/DL, etc.).” from a nice article on it, PyScript — unleash the power of Python in your browser
* PyScript is built on Pyodide, which is a port of CPython based on WebAssembly.
* Demos are cool.
* Note included in README: “This is an extremely experimental project, so expect things to break!”
Michael #2: Memray from Bloomberg
* Memray is a memory profiler for Python.
* It can track memory allocations in
* Python code
* native extension modules
* the Python interpreter itself
* Works both via CLI and focused app calls
* Memray can help with the following problems:
* Analyze allocations in applications to help discover the cause of high memory usage.
* Find memory leaks.
* Find hotspots in code which cause a lot of allocations.
* Notable features:
* 🕵️♀️ Traces every function call so it can accurately represent the call stack, unlike sampling profilers.
* ℭ Also handles native calls in C/C++ libraries so the entire call stack is present in the results.
* 🏎 Blazing fast! Profiling causes minimal slowdown in the application. Tracking native code is somewhat slower, but this can be enabled or disabled on demand.
* 📈 It can generate various reports about the collected memory usage data, like flame graphs.
* 🧵 Works with Python threads.
* 👽🧵 Works with native-threads (e.g. C++ threads in native extensions)
* Has a live view in the terminal.
* Linux only
Brian #3: pytest-parallel
* I’ve often sped up tests that can be run in parallel by using -n from pytest-xdist.
* I was recommending this to someone on Twitter, and Bruno Oliviera suggested a couple of alternatives. One was pytest-parallel, so I gave it a try.
* pytest-xdist runs using multiprocessing
* pytest-parallel uses both multiprocessing and multithreading.
* This is especially useful for test suites containing threadsafe tests. That is, mostly, pure software tests.
* Lots of unit tests are like this. System tests are often not.
* Use --workers flag for multiple processors, --workers auto works great.
* Use --tests-per-worker for multi-threading. --tesst-per-worker auto let’s it pick.
* Very cool alternative to xdist. -
Michael #4: Pooch: A friend for data files
* via via Matthew Fieckert
* Just want to download a file without messing with requests and urllib?
* Who is it for? Scientists/researchers/developers looking to simply download a file.
* Pooch makes it easy to download a file (one function call). On top of that, it also comes with some bonus features:
* Download and cache your data files locally (so it’s only downloaded once).
* Make sure everyone running the code has the same version of the data files by verifying cryptographic hashes.
* Multiple download protocols HTTP/FTP/SFTP and basic authentication.
* Download from Digital Object Identifiers (DOIs) issued by repositories like figshare and Zenodo.
* Built-in utilities to unzip/decompress files upon download
* file_path = pooch.retrieve(url)
Extras
Michael:
* New course! Up and Running with Git - A Pragmatic, UI-based Introduction.
Joke:
* Don’t embarrass me in front of the wizards
* Michael’s crashing github is embarrassing him in front of the wizards!
playlist_add
Apr 28, 2022
#281 ohmyzsh + ohmyposh + mcfly + pls + nerdfonts = wow
Watch the live stream: Watch on YouTube
About the show
Sponsored: RedHat: Compiler Podcast
Special guest: Anna Astori
Michael #1: Take Your Github Repository To The Next Level 🚀️
* Step 0. Make Your Project More Discoverable
* Step 1. Choose A Name That Sticks
* Step 2. Display A Beautiful Cover Image
* Step 3. Add Badges To Convey More Information
* Step 4. Write A Convincing Description
* Step 5. Record Visuals To Attract Users 👀
* Step 6. Create A Detailed Installation Guide (if needed)
* Step 7. Create A Practical Usage Guide 🏁
* Step 8. Answer Common Questions
* Step 9. Build A Supportive Community
* Step 10. Create Contribution Guidelines
* Step 11. Choose The Right License
* Step 12. Plan Your Future Roadmap
* Step 13. Create Github Releases (know release drafter)
* Step 14. Customize Your Social Media Preview
* Step 15. Launch A Website
Brian #2: Fastero
* “Python timeit CLI for the 21st century.”
* Arian Mollik Wasi, @wasi_master
* Colorful and very usable benchmarking/comparison tool
* Time or Compare one ore more
* code snippet
* python file
* mix and match, even
* Allows setup code before snippets run
* Multiple output export formats: markdown, html, csv, json, images, …
* Lots of customization possible
* Takeaway
* especially for comparing two+ options, this is super handy
Anna #3:
* langid vs langdetect
langdetect
* This library is a direct port of Google's language-detection library from Java to Python
* langdetect supports 55 languages out of the box (ISO 639-1 codes):
* Basic usage: detect() and detect_langs()
* great to work with noisy data like social media and web blogs
* being statistical, works better on larger pieces of text vs short posts
langid
* hasn't been updated for a few years
* 97 languages
* can use Python's built-in wsgiref.simple_server (or fapws3 if available) to provide language identification as a web service. To do this, launch python langid.py -s, and access http://localhost:9008/detect . The web service supports GET, POST and PUT.
* the actual calculations are implemented in the log-probability space but can also have a "confidence" score for the probability prediction between 0 and 1: > from langid.langid import LanguageIdentifier, model > identifier = LanguageIdentifier.from_modelstring(model, norm_probs=True) > identifier.classify("This is a test") > ('en', 0.9999999909903544) - minimal dependencies - relatively fast - NB algo, can train on user data.
Michael #4: Watchfiles
* by Samual Colvin (of Pydantic fame)
* Simple, modern and high performance file watching and code reload in python.
* Underlying file system notifications are handled by the Notify rust library.
* Supports sync watching but also async watching
* CLI example
* Running and restarting a command¶
* Let's say you want to re-run failing tests whenever files change. You could do this with watchfiles using
* Running a command: watchfiles 'pytest --lf``'
Brian #5: Slipcover: Near Zero-Overhead Python Code Coverage
* From coverage.py twitter account, which I’m pretty sure is Ned Bachelder
* coverage numbers with “3% or less overhead”
* Early stages of the project.
* It does seem pretty zippy though.
* Mixed results when trying it out with a couple different projects
* flask:
* just pytest: 2.70s
* with slipcover: 2.88s
* with coverage.py: 4.36s
* flask with xdist n=4
* pytest: 2.11 s
* coverage: 2.60s
* slipcover: doesn’t run (seems to load pytest plugins)
* Again, still worth looking at and watching. It’s good to see some innovation in the coverage space aside from Ned’s work.
Anna #6:
* scrapy vs robox
scra-py
* shell to try out things: fetch url, view response object, response.text
* extract using css selectors or xpath
* lets you navigate between levels e.g. the parent of an element with id X
* crawler to crawl websites and spider to extract data
* startproject for project structure and templates like settings and pipelines
* some advanced features like specifying user-agents etc for large scale scraping.
* various options to export and store the data
* nice features like LinkExtractor to determine specific links to extract, already deduped.
* FormRequest class
robox
* layer on top of httpx and beautifulsoup4
* allows to interact with forms on pages: check, choose, submit
Extras
Michael:
* ohmyzsh + ohmyposh + mcfly + iterm2 + pls + nerdfonts = wow
* Watch the video we discussed here
Joke:
* Out for a byte
playlist_add