Ducktools: Class Builder

ducktools-classbuilder is the Python package that will bring you the joy of writing... functions... that will bring back the joy of writing classes.

Maybe.

While attrs and dataclasses are class boilerplate generators, ducktools.classbuilder is intended to provide the tools to help make a customized version of the same concept.

Install from PyPI with: python -m pip install ducktools-classbuilder

Included Implementations

The classbuilder tools make up the core of this module and there is an implementation using these tools in the prefab submodule.

There is also a minimal @slotclass example that can construct classes from a special mapping used in __slots__.

from ducktools.classbuilder import Field, SlotFields, slotclass

@slotclass
class SlottedDC:
    __slots__ = SlotFields(
        the_answer=42,
        the_question=Field(
            default="What do you get if you multiply six by nine?",
            doc="Life, the Universe, and Everything",
        ),
    )

ex = SlottedDC()
print(ex)

Core

The core of the module provides tools for creating a customized version of the dataclass concept.

• MethodMaker
  • This tool takes a function that generates source code and converts it into a descriptor that will execute the source code and attach the gemerated method to a class on demand.
• Field
  • This defines a basic dataclass-like field with some basic arguments
  • This class itself is a dataclass-like of sorts
  • Additional arguments can be added by subclassing and using annotations
    • See ducktools.classbuilder.prefab.Attribute for an example of this
• Gatherers
  • These collect field information and return both the gathered fields and any modifications that will need to be made to the class when built to support them.
• builder
  • This is the main tool used for constructing decorators and base classes to provide generated methods.
  • Other than the required changes to a class for __slots__ that are done by SlotMakerMeta this is where all class mutations should be applied.
• SlotMakerMeta
  • When given a gatherer, this metaclass will create __slots__ automatically.

Tip

For more information on using these tools to create your own implementations using the builder see the tutorial for a full tutorial and extension_examples for other customizations.

Prefab

This prebuilt implementation is available from the ducktools.classbuilder.prefab submodule.

This includes more customization including __prefab_pre_init__ and __prefab_post_init__ functions for subclass customization.

A @prefab decorator and Prefab base class are provided.

Prefab will generate __slots__ by default. decorated classes with @prefab that do not declare fields using __slots__ will not be slotted and there is no slots argument to apply this.

Here is an example of applying a conversion in __post_init__:

from pathlib import Path
from ducktools.classbuilder.prefab import Prefab

class AppDetails(Prefab, frozen=True):
    app_name: str
    app_path: Path

    def __prefab_post_init__(self, app_path: str | Path):
        # frozen in `Prefab` is implemented as a 'set-once' __setattr__ function.
        # So we do not need to use `object.__setattr__` here
        self.app_path = Path(app_path)

steam = AppDetails(
    "Steam",
    r"C:\Program Files (x86)\Steam\steam.exe"
)

print(steam)

Features

Prefab and @prefab support many standard dataclass features along with a few extras.

• All standard methods are generated on-demand
  • This makes the construction of classes much faster in general
  • Generation is done and then cached on first access
• Standard __init__, __eq__ and __repr__ methods are generated by default
  • The __repr__ implementation does not automatically protect against recursion, but there is a recursive_repr argument that will do so if needed
• repr, eq and kw_only arguments work as they do in dataclasses
• There is an optional iter argument that will make the class iterable
• __prefab_post_init__ will take any field name as an argument and can be used to write a 'partial' __init__ function for only non-standard attributes
• The frozen argument will make the dataclass a 'write once' object
  • This is to make the partial __prefab_post_init__ function more natural to write for frozen classes
• dict_method=True will generate an as_dict method that gives a dictionary of attributes that have serialize=True (the default)
• ignore_annotations can be used to only use the presence of attribute values to decide how the class is constructed
  • This is intended for cases where evaluating the annotations may trigger imports which could be slow and unnecessary for the function of class generation
• replace=False can be used to avoid defining the __replace__ method
• attribute has additional options over dataclasses' Field
  • iter=True will include the attribute in the iterable if __iter__ is generated
  • serialize=True decides if the attribute is include in as_dict
  • exclude_field is short for repr=False, compare=False, iter=False, serialize=False
  • private is short for exclude_field=True and init=False and requires a default/factory
  • doc will add this string as the value in slotted classes, which appears in help()
• build_prefab can be used to dynamically create classes and does support a slots argument
  • Unlike dataclasses, this does not create the class twice in order to provide slots

There are also some intentionally missing features:

• The @prefab decorator does not and will not support a slots argument
  • Use Prefab for slots.
• as_dict and the generated .as_dict method do not recurse or deep copy
• unsafe_hash is not provided
• weakref_slot is not available as an argument
  • __weakref__ can be added to slots by declaring it as if it were an attribute
• There is no check for mutable defaults
  • You should still use default_factory as you would for dataclasses, not doing so is still incorrect
  • dataclasses uses hashability as a proxy for mutability, but technically this is inaccurate as you can be unhashable but immutable and mutable but hashable
  • This may change in a future version, but I haven't felt the need to add this check so far
• In Python 3.14 Annotations are gathered as VALUE if possible and STRING if this fails
  • VALUE annotations are used as they are faster in most cases
  • As the __init__ method gets __annotations__ these need to be either values or strings to match the behaviour of previous Python versions

What is the issue with generating __slots__ with a decorator

If you want to use __slots__ in order to save memory you have to declare them when the class is originally created as you can't add them later.

When you use @dataclass(slots=True)¹ with dataclasses, the function has to make a new class and attempt to copy over everything from the original.

This is because decorators operate on classes after they have been created while slots need to be declared beforehand. While you can change the value of __slots__ after a class has been created, this will have no effect on the internal structure of the class.

By using a metaclass or by declaring fields using __slots__ however, the fields can be set before the class is constructed, so the class will work correctly without needing to be rebuilt.

For example these two classes would be roughly equivalent, except that @dataclass has had to recreate the class from scratch while Prefab has created __slots__ and added the methods on to the original class. This means that any references stored to the original class before @dataclass has rebuilt the class will not be pointing towards the correct class.

Here's a demonstration of the issue using a registry for serialization functions.

This example requires Python 3.10 or later as earlier versions of dataclasses did not support the slots argument.

import json
from dataclasses import dataclass
from ducktools.classbuilder.prefab import Prefab, attribute


class _RegisterDescriptor:
    def __init__(self, func, registry):
        self.func = func
        self.registry = registry

    def __set_name__(self, owner, name):
        self.registry.register(owner, self.func)
        setattr(owner, name, self.func)


class SerializeRegister:
    def __init__(self):
        self.serializers = {}

    def register(self, cls, func):
        self.serializers[cls] = func

    def register_method(self, method):
        return _RegisterDescriptor(method, self)

    def default(self, o):
        try:
            return self.serializers[type(o)](o)
        except KeyError:
            raise TypeError(f"Object of type {type(o).__name__} is not JSON serializable")


register = SerializeRegister()


@dataclass(slots=True)
class DataCoords:
    x: float = 0.0
    y: float = 0.0

    @register.register_method
    def to_json(self):
        return {"x": self.x, "y": self.y}


# slots=True is the default for Prefab
class BuilderCoords(Prefab):
    x: float = 0.0
    y: float = attribute(default=0.0, doc="y coordinate")

    @register.register_method
    def to_json(self):
        return {"x": self.x, "y": self.y}


# In both cases __slots__ have been defined
print(f"{DataCoords.__slots__ = }")
print(f"{BuilderCoords.__slots__ = }\n")

data_ex = DataCoords()
builder_ex = BuilderCoords()

objs = [data_ex, builder_ex]

print(data_ex)
print(builder_ex)
print()

# Demonstrate you can not set values not defined in slots
for obj in objs:
    try:
        obj.z = 1.0
    except AttributeError as e:
        print(e)
print()

print("Attempt to serialize:")
for obj in objs:
    try:
        print(f"{type(obj).__name__}: {json.dumps(obj, default=register.default)}")
    except TypeError as e:
        print(f"{type(obj).__name__}: {e!r}")

Output (Python 3.12):

DataCoords.__slots__ = ('x', 'y')
BuilderCoords.__slots__ = {'x': None, 'y': 'y coordinate'}

DataCoords(x=0.0, y=0.0)
BuilderCoords(x=0.0, y=0.0)

'DataCoords' object has no attribute 'z'
'BuilderCoords' object has no attribute 'z'

Attempt to serialize:
DataCoords: TypeError('Object of type DataCoords is not JSON serializable')
BuilderCoords: {"x": 0.0, "y": 0.0}

Will you add <feature> to classbuilder.prefab?

No. Not unless it's something I need or find interesting.

The original version of prefab_classes was intended to have every feature anybody could possibly require, but this is no longer the case with this rebuilt version.

I will fix bugs (assuming they're not actually intended behaviour).

However the whole goal of this module is if you want to have a class generator with a specific feature, you can create or add it yourself.

Credit

Heavily inspired by David Beazley's Cluegen

Footnotes

1. or @attrs.define. ↩