Wrappers and Extenders

Companion Packages for Python Projects

Jules Walzer-Goldfeld

About Me

• Software developer, tool builder
• Author of gt-extras
• Great Tables at Posit: Michael Chow and Richard Iannone
• I enjoy 🏓, ⚽, 🧗, 🚴

• turn on DND or focus mode
• Thank you all for coming
• Don’t need to know about great tables for this talk

Biking

Roommate Story

• Four roommates that share a bike
• Bike has an owner

Maintenance, Upgrades

• Owner is great
• Maintains lights, bells and whistles, drivetrain

You Borrow It

• You want to borrow the bike
• Your use case is different
• Bring lunch or mouse
• Main user isn’t maintaining the bike for the purpose

Solve It For Yourself

• Do it yourself
• Back sweat
• Other people borrow bike, can’t take your bag

Modify The Bike For Everyone

• Don’t need a bag
• Owner has to agree to this
• Makes the bike heavier

Fork The Bike

• Copy the bike
• We copy owner’s upgrades
• Rear fender upgrade not compatible with basket

Use An Optional Attachment

• Holy grail
• Dream of mine
• Removable, therefore optional
• Other roommates can use it
• Won’t interfere with owner
• Don’t have to maintain your own bike

Developing Packages

• Should be at 7:30

• Marinate on how this has to do with developing packages

Programming story

• Michael and Richard and my co-workers/roommates
• Great tables = dataframes (pandas) to publication tables
• Cheatsheet outside

Maintenance, Upgrades

• Always adding new features
• Responding and working with community
• Akin to the bike owner

You Borrow It

• Can’t change tabs from here
• Using the bike
• Lots of text, want plots instead
• Dumbbell and bar plots
• Not everyone wants/needs this

• The Dataset
• Great Table
• Your Great(er) Table

shape: (9, 11)

Entity	Code	Year	gini_disposable__age_total	gini_market__age_total	population_historical	owid_region	gini_pct_change	gini_pct_benchmark_5yr	pop_log	pop_icons
str	str	i64	f64	f64	str	str	f64	f64	f64	i64
"France"	"FRA"	2020	0.278	0.515	"65,905,226"	"Europe"	46.02	null	7.81892	3
"Germany"	"DEU"	2020	0.303	0.496	"83,628,661"	"Europe"	38.91	40.966	7.922355	4
"South Korea"	"KOR"	2020	0.328	0.405	"51,858,440"	"Asia"	19.01	14.756	7.714819	1
"Spain"	"ESP"	2020	0.329	0.515	"47,679,437"	"Europe"	36.12	34.792	7.678331	1
"Italy"	"ITA"	2020	0.331	0.527	"59,912,714"	"Europe"	37.19	36.08	7.777519	2
"United Kingdom"	"GBR"	2020	0.355	0.507	"67,351,806"	"Europe"	29.98	29.332	7.828349	3
"United States"	"USA"	2020	0.377	0.521	"339,436,106"	"North America"	27.64	23.48	8.530758	11
"Turkey"	"TUR"	2020	0.403	0.504	"86,091,644"	"Asia"	20.04	16.208	7.934961	4
"Mexico"	"MEX"	2020	0.42	0.435	"126,798,998"	"North America"	3.45	3.35	8.103116	6

Income Inequality Before and After Taxes in 2020
		gini_disposable__age_total	gini_market__age_total	Population	Improvement Post Taxes	gini_pct_benchmark_5yr
Europe
France		0.278	0.515		46.02	None
Germany		0.303	0.496		38.91	40.966
Spain		0.329	0.515		36.12	34.791999999999994
Italy		0.331	0.527		37.19	36.080000000000005
United Kingdom		0.355	0.507		29.98	29.332
Asia
South Korea		0.328	0.405		19.01	14.756
Turkey		0.403	0.504		20.04	16.208
North America
United States		0.377	0.521		27.64	23.48
Mexico		0.42	0.435		3.45	3.35

Income Inequality Before and After Taxes in 2020
		Pre-tax to Post-tax Coefficient	Population	Improvement Post Taxes
Europe
France			65,905,226
Germany			83,628,661
Spain			47,679,437
Italy			59,912,714
United Kingdom			67,351,806
Asia
South Korea			51,858,440
Turkey			86,091,644
North America
United States			339,436,106
Mexico			126,798,998

Solve It For Yourself

Write It Locally

• Bring your own backpack
• Build in a script, locally
• fmt_nanoplot doesn’t do our plotting
• fmt lets you plug in generically (show 2nd image)
• Hard to share with peers or collaborate
  • great starting point

Modify The Bike For Everyone

Submit a PR For Everyone

• Take that local script
  • make a function
  • submit PR
  • Have to learn about internals
• might end up waiting
• might get heavily edited
  • (hopefully not issues in great tables)

Fork The Bike

Fork the Package

• Good for sharing
• Shadow-maintaining the package
  • Watch for updates
  • Check for breaking
• Users have to rely on you for work that isn’t yours

Use An Optional Attachment

Build A Companion Package

• gt-extras + great_tables
• nbmail or red-mail + smtplib
• requests-toolbelt + requests

• Our saddle bag of packages
  • not required for great tables use
  • Anyone can use
  • don’t need your own
• redmail, yagmail wrap
• requests-toolbelt for data-scraping

What Goes In The Package

• Should be at 16:00

What’s In A Package?

• Collection of actions
  • gt is dataframe -> beautiful table
• Been talking packages too long

1. Many options in one selection, easy access to one
   • click button to set this specific gear
2. Many places to make many selections, easy access to a set
   • 3rd gear, Lights, Reflective vest
3. New functionality not supported
   • Re-structure the bike
   • Don’t fold a road bike

1. 17th gear
2. Night mountain ride
3. Folding bike

1. Wrapping
2. Composing
3. Extending

1) Wrapping

from great_tables import GT, html
from great_tables.data import airquality

airquality_mini = airquality.head(10).assign(Year=1973)

gt = (
    GT(airquality_mini)
    .tab_header(
        title="New York Air Quality Measurements",
        subtitle="Daily measurements in New York City (May 1-10, 1973)"
    )
    .tab_spanner(label="Time", columns=["Year", "Month", "Day"])
    .tab_spanner(
        label="Measurement",
        columns=["Ozone", "Solar_R", "Wind", "Temp"],
    )
    .cols_move_to_start(columns=["Year", "Month", "Day"])
    .cols_label(
        Ozone=html("Ozone,<br>ppbV"),
        Solar_R=html("Solar R.,<br>cal/m<sup>2</sup>"),
        Wind=html("Wind,<br>mph"),
        Temp=html("Temp,<br>&deg;F"),
    )
)

gt

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	7.4	67
1973	5	2	36.0	118.0	8.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	8.6	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	8.6	69

• Example dataset
• Air quality measures May 1973

1) Wrapping

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	7.4	67
1973	5	2	36.0	118.0	8.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	8.6	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	8.6	69

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	7.4	67
1973	5	2	36.0	118.0	8.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	8.6	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	8.6	69

• I want to highlight
• Can do with great tables

1) Wrapping

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	7.4	67
1973	5	2	36.0	118.0	8.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	8.6	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	8.6	69

from great_tables import loc, style

locations = [
    loc.body(columns="Wind"),
    loc.column_labels(columns="Wind"),
]

# Make the tab_style call
gt.tab_style(
    style=style.fill(color="lightblue"),
    locations=locations,
)

from great_tables import GT, loc, style

def highlight_columns(gt: GT, columns) -> GT:
    locations = [
        loc.body(columns=columns),
        loc.column_labels(columns=columns),
    ]

    # Make the wrapped tab_style call
    return gt.tab_style(
        style=style.fill(color="lightblue"),
        locations=locations,
    )

highlight_columns(gt, columns="Wind")

• How we would do it in great tables
• Wrap it in our own function
• Now can call it in one line
• tab_style has many options
  • we wrapped it and chose one
  • fewer lines

2) Composing

from great_tables import GT, html
from great_tables.data import airquality

airquality_mini = airquality.head(10).assign(Year=1973)

gt = (
    GT(airquality_mini)
    .tab_header(
        title="New York Air Quality Measurements",
        subtitle="Daily measurements in New York City (May 1-10, 1973)"
    )
    .tab_spanner(label="Time", columns=["Year", "Month", "Day"])
    .tab_spanner(
        label="Measurement",
        columns=["Ozone", "Solar_R", "Wind", "Temp"],
    )
    .cols_move_to_start(columns=["Year", "Month", "Day"])
    .cols_label(
        Ozone=html("Ozone,<br>ppbV"),
        Solar_R=html("Solar R.,<br>cal/m<sup>2</sup>"),
        Wind=html("Wind,<br>mph"),
        Temp=html("Temp,<br>&deg;F"),
    )
)

gt

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	7.4	67
1973	5	2	36.0	118.0	8.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	8.6	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	8.6	69

• restart
• Same dataset

2) Composing

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	7.4	67
1973	5	2	36.0	118.0	8.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	8.6	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	8.6	69

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	7.4	67
1973	5	2	36.0	118.0	8.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	8.6	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	8.6	69

• Table theme
  • font
  • spacing
  • lowercased

2) Composing

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	7.4	67
1973	5	2	36.0	118.0	8.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	8.6	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	8.6	69

def some_theme(gt: GT) -> GT:
    return (
        gt
        .opt_row_striping()
        .opt_table_font(font="Courier")
        .tab_options(
            heading_align="left",
            column_labels_text_transform="lowercase",
            row_striping_background_color="#b5dbb6",
        )
    )

gt.pipe(some_theme)

• Compose many choices
• can call some theme and pass in object
  • or use the pipe method
• Can apply theme to another table

3) Extending

from great_tables import GT, html
from great_tables.data import airquality

airquality_mini = airquality.head(10).assign(Year=1973)

gt = (
    GT(airquality_mini)
    .tab_header(
        title="New York Air Quality Measurements",
        subtitle="Daily measurements in New York City (May 1-10, 1973)"
    )
    .tab_spanner(label="Time", columns=["Year", "Month", "Day"])
    .tab_spanner(
        label="Measurement",
        columns=["Ozone", "Solar_R", "Wind", "Temp"],
    )
    .cols_move_to_start(columns=["Year", "Month", "Day"])
    .cols_label(
        Ozone=html("Ozone,<br>ppbV"),
        Solar_R=html("Solar R.,<br>cal/m<sup>2</sup>"),
        Wind=html("Wind,<br>mph"),
        Temp=html("Temp,<br>&deg;F"),
    )
)

gt

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	7.4	67
1973	5	2	36.0	118.0	8.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	8.6	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	8.6	69

• Things not possible in great tables

3) Extending

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	7.4	67
1973	5	2	36.0	118.0	8.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	8.6	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	8.6	69

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	<10.0	67
1973	5	2	36.0	118.0	<10.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	<10.0	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	<10.0	69

• Plotting, dumbbell or bar
• Truncate, might be nice for p-value

3) Extending

New York Air Quality Measurements
Daily measurements in New York City (May 1-10, 1973)
Time			Measurement
Year	Month	Day	Ozone, ppbV	Solar R., cal/m2	Wind, mph	Temp, °F
1973	5	1	41.0	190.0	<10.0	67
1973	5	2	36.0	118.0	<10.0	72
1973	5	3	12.0	149.0	12.6	74
1973	5	4	18.0	313.0	11.5	62
1973	5	5			14.3	56
1973	5	6	28.0		14.9	66
1973	5	7	23.0	299.0	<10.0	65
1973	5	8	19.0	99.0	13.8	59
1973	5	9	8.0	19.0	20.1	61
1973	5	10		194.0	<10.0	69

def fmt_threshold(gt, columns, threshold: float):

    def _truncate(val: float):
        if val < threshold:
            return f"<{threshold}"
        else:
            return str(val)

    return gt.fmt(_truncate, columns)

gt.pipe(fmt_threshold, "Wind", 10.0)

• _truncate does the heavy lifting
  • could do the same with any svg, plot
• Wrap gt.fmt with the new functionality

Finishing the Package

• Should be at 24:00

• You’ve written all of the desired utilities

• Ideally version controlled

• How to bundle them into a package

Package Structure

Metadata and source code

gt-extras
├── pyproject.toml            # Metadata
└── gt_extras/                # Source code
    ├── __init__.py
    └── functions.py

• At this point assume we have the gt_extras source code
• __init__ exports the functions
• Add some metadata
• This is just one approach

Package Structure

Tests

gt-extras
├── pyproject.toml            # Metadata
└── gt_extras/                # Source code
    ├── __init__.py
    ├── functions.py
    └── tests/                # Package tests

• Tests ensure upstream changes don’t break our package

Package Structure

Documentation

gt-extras
├── docs/                     # Package documentation
│   ├── getting-started.md
│   ├── examples.md
│   └── gt_extras_reference.md
├── LICENSE
├── README.md
├── Makefile                  # Metadata
├── pyproject.toml
└── gt_extras/                # Source code
    ├── __init__.py
    ├── functions.py
    └── tests/                # Package tests

• Documentation helps users engage

Package Structure

Continuous Integration / Continuous Development

gt-extras
├── .github/                  # CI/CD
│   ├── CONTRIBUTING.md
│   └── workflows/
│       ├── ci_build.yml
│       └── ci_tests.yml
├── docs/                     # Package documentation
│   ├── getting-started.md
│   ├── examples.md
│   └── gt_extras_reference.md
├── LICENSE
├── README.md
├── Makefile                  # Metadata
├── pyproject.toml
└── gt_extras/                # Source code
    ├── __init__.py
    ├── functions.py
    └── tests/                # Package tests

• Use a pipeline, github workflows can do this

Tests

Using pytest

import pytest
from great_tables import GT
from gt_extras import highlight_columns

• Show two possible tests

Tests

Snapshot test

import pytest
from great_tables import GT
from gt_extras import highlight_columns

   
def test_highlight_columns_snap(snapshot, mini_gt):
    res = highlight_columns(mini_gt)
    assert_rendered_body(snapshot, gt=res)

• Snapshot tests
  • Saves raw output to disc
  • Might not be ideal with massive data
  • Easy to parse as developer

Tests

Check content directly

import pytest
from great_tables import GT
from gt_extras import highlight_columns

   
def test_highlight_columns_snap(snapshot, mini_gt):
    res = highlight_columns(mini_gt)
    assert_rendered_body(snapshot, gt=res)


def test_highlight_columns(mini_gt):
    gt = highlight_columns(mini_gt, columns=[1, 2])
    html = gt.as_raw_html()
    assert html.count("lightblue") == num_rows(mini_gt) * 2

• Check analytically
• Can pass in different tables

Documentation

Docstring

def highlight_columns(gt: GT, columns) -> GT:
    """
    Add color highlighting to one or more specific columns.

    Parameters
    ----------
    gt
        An existing `GT` object.

    columns
        The columns to target. Can be a single column or a list of columns (by name or index).
        If `None`, the coloring is applied to all columns.

• Many many approaches
• This docstring gets parsed by quartodoc
  • Produces webpages for API docs
• Title, params

Documentation

Return value

def highlight_columns(gt: GT, columns) -> GT:
    """
    Add color highlighting to one or more specific columns.

    Parameters
    ----------
    gt
        An existing `GT` object.

    columns
        The columns to target. Can be a single column or a list of columns (by name or index).
        If `None`, the coloring is applied to all columns.

    Returns
    -------
    GT
        The `GT` object is returned.

• Return value

Documentation

Example

    """
    Parameters
    ----------
    gt
        An existing `GT` object.

    columns
        The columns to target. Can be a single column or a list of columns (by name or index).
        If `None`, the coloring is applied to all columns.

    Returns
    -------
    GT
        The `GT` object is returned.

    Example
    --------
    gt.pipe(gte.highlight_columns, columns="hp")
    """

• Return value
• Examples
• Michael Chow has an awesome talk on this very topic
  • quartodoc is his library

CI/CD

Continuous Integration / Continuous Development

.github/workflows/*.yml

• .yaml file to execute jobs

CI/CD

When to run it

name: Tests

on: [push, pull_request, release]

• .yaml file to execute jobs
• Happens on certain triggers

CI/CD

Write a job

name: Tests

on: [push, pull_request, release]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.9', '3.12']
    steps:
      - name: Checkout
      - name: Install uv
      - name: Set up Python
      - name: Install the project
      - name: Test

• .yaml file to execute jobs
• Happens on certain triggers
• Runs on certain test environments
• Performs a number of actions
  • checking failures

CI/CD

Execute each step

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Install uv
        uses: astral-sh/setup-uv@v5
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install the project
        run: uv sync --all-extras --dev
      - name: Test
        run: |
          uv run pytest

• Calling common github actions
• Executing bash content
  • install dependencies
  • run tests

Configuration

pyproject.toml

[project]
name = "gt-extras"
version = "0.1.0"
description = "Provides additional functions for creating beautiful tables with 'great-tables'."

• Metadata

Configuration

Requirements

[project]
name = "gt-extras"
version = "0.1.0"
description = "Provides additional functions for creating beautiful tables with 'great-tables'."
requires-python = ">=3.9"
dependencies = [
    "great-tables>=0.18.0",
    "svg-py>=1.6.0",
]

• Metadata
• Dependencies
  • Plot graphics

Configuration

Other metadata

[project]
name = "gt-extras"
version = "0.1.0"
description = "Provides additional functions for creating beautiful tables with 'great-tables'."
requires-python = ">=3.9"
dependencies = [
    "great-tables>=0.18.0",
    "svg-py>=1.6.0",
]

[tool.pytest.ini_options]
minversion = "6.0"
addopts = "-ra --cov=gt_extras --cov-report=term-missing"
testpaths = ["gt_extras/tests"]

keywords = ["tables", "html"]

• Metadata
• Dependencies
  • Plot graphics
• More metadata
  • How to run tests
  • Keywords

We Talked About

• Bicycle bags!

We Talked About

• Bicycle bags!
• Why to write a companion package

We Talked About

• Bicycle bags!
• Why to write a companion package
• How to write a companion package

We Talked About

• Bicycle bags!
• Why to write a companion package
• How to write a companion package
• How to build a companion package

Thank You

to Michael Chow and Richard Iannone (and the Posit family)

• Great Tables

• gt-extras