Watch the live stream:
Watch on YouTube
About the show
Sponsored by us:
Special guest: Erik Christiansen
Michael #1: Fickling
- via Oli
- A Python pickling decompiler and static analyzer
- Pickled ML models are becoming the data exchange and workflow of ML
- Analyses pickle files for security risks - It can also remove or insert [malicious] code into pickle files...
- Created by a security firm, it can be a useful defensive or offensive tool.
- Perhaps it is time to screen all pickles?
>>> import ast
>>> import pickle
>>> from fickling.pickle import Pickled
>>> print(ast.dump(Pickled.load(pickle.dumps([1, 2, 3, 4])).ast, indent=4))
- You can test for common patterns of malicious pickle files with the
- You can also safely trace the execution of the Pickle virtual machine without exercising any malicious code with the
- Finally, you can inject arbitrary Python code that will be run on unpickling into an existing pickle file with the
- See Risky Biz's episode for more details.
Brian #2: Python Project-Local Virtualenv Management
- Hynek Schlawack
- Only works on UNIX-like systems. MacOS, for example.
- Install direnv. (ex: brew install direnv)
- Put this into a
.envrc file in your project root:
layout python python3.9
- when you
cd into that directory or a subdirectory, your virtual environment is loaded.
- when you cd out of it, the venv is unloaded
- Michael covered direnv on Episode 185. But it wasn’t until Hynek spelled it out for me how to use it with venv that I understood the simplicity and power.
- Not really faster than creating a venv, but when flipping between several projects, it’s way faster than deactivating/activating.
- You can also set env variables per directory (kinda the point of direnv)
Erik #3: Testcontainers
“Python port for testcontainers-java that allows using docker containers for functional and integration testing. Testcontainers-python provides capabilities to spin up docker containers (such as a database, Selenium web browser, or any other container) for testing. “ (pypi description).
- Provides cloud native services, many databases and the like (e.g. Google Cloud Pub/Sub, Kafka..)
- Originally a java project, still a way to go for us python programmers to implement all services
- Provides an example for use in CI/CD by leveraging Docker in Docker
from testcontainers.mysql import MySqlContainer
with MySqlContainer('mysql:5.7.17') as mysql:
engine = sqlalchemy.create_engine(mysql.get_connection_url())
version, = engine.execute("select version()").fetchone()
print(version) # 5.7.17
Michael #4: jc
- via Garett
- CLI tool and python library that converts the output of popular command-line tools and file-types to JSON or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.
- Run it as
COMMAND ARGS | jc --COMMAND
- Commands include:
Brian #5: What is Python's Ellipsis Object?
- Florian Dahlitz
… is a constant defined in Python
- “Ellipsis: The same as the ellipsis literal “...”. Special value used mostly in conjunction with extended slicing syntax for user-defined container data types.”
- Can be used in type hinting
- Func returns two int tuple
def return_tuple() -> tuple[int, int]:
- Func returns one or more integer:
def return_tuple() -> tuple[int, ...]:
- Ellipsis in the wild, “if you want to implement a certain feature where you need a non-used literal, you can use the ellipsis object.”
Erik #6: PyTorch Forecasting
PyTorch Forecasting aims to ease state-of-the-art timeseries forecasting with neural networks for both real-world cases and research alike. The goal is to provide a high-level API with maximum flexibility for professionals and reasonable defaults for beginners.
- basically tries to achieve for time series what fast.ai has achieved for computer vision and natural language processing
- The package is built on PyTorch Lightning to allow training on CPUs, single and multiple GPUs out-of-the-box.
- Implements of Temporal Fusion Transformers
- interpretable - can calculate feature importance
- Hyperparameter tuning with optuna
Joke: 200 == 400