Add __slots__ to lightweight connection classes for memory optimization #1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces
__slots__to 12 frequently-instantiated lightweight classes incassandra/connection.pyto reduce per-instance memory overhead and improve performance. This is the first phase of memory optimization improvements as discussed previously.Motivation
Many small objects in connection.py are created per connection or per request cycle. Each Python object without
__slots__maintains a per-instance__dict__(and sometimes a__weakref__) that adds significant overhead (~56+ bytes depending on Python build). For frequently instantiated objects like endpoints, frames, and utility classes, this overhead can accumulate to substantial memory usage.Changes
Added
__slots__to the following classes with their respective slot definitions:()- Base class with empty slots for consistency('_address', '_port')('_proxy_address', '_index', '_resolved_address', '_port', '_server_name', '_ssl_options')('_unix_socket_path',)('version', 'flags', 'stream', 'opcode', 'body_offset', 'end_pos')('stream_id', 'decoder', 'row_factory', 'connection', '_condition', '_stop', '_page_queue', '_state', 'released')()- Utility class with classmethods only('_io_buffer', '_cql_frame_buffer', '_connection', '_segment_consumed')('connection', 'pending', 'fail_on_error', 'error', 'responses', 'event')('_exception', '_event', 'connection', 'owner')('end', 'callback', 'canceled')('_queue', '_new_timers')Technical Details
canceled = Falsefrom class variable to proper instance initialization to work correctly with__slots____init__Benefits
__dict__overhead, reducing memory footprint by ~56+ bytes per objectTesting
tests/unit/test_connection_slots.py) to verify__slots__implementation__dict__attributesAttributeErrorCompatibility
This is a purely internal memory optimization with no breaking changes. All existing code will continue to work exactly as before, but with reduced memory overhead for these lightweight objects.
Future Work
This sets the foundation for subsequent memory optimization PRs that can:
Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
proxy.example.comTest inheritance behavior with slots
from cassandra.connection import EndPoint, DefaultEndPoint, SniEndPoint, UnixSocketEndPoint
print('=== Inheritance and Polymorphism Tests ===')
Create instances of all endpoint types
endpoints = [
DefaultEndPoint('127.0.0.1', 9042),
SniEndPoint('proxy.example.com', 'server.example.com', 9042),
UnixSocketEndPoint('/tmp/cassandra.sock')
]
Test that they are all instances of EndPoint
for ep in endpoints:
print(f'{ep.class.name}: isinstance(EndPoint)={isinstance(ep, EndPoint)}, address={ep.address}')
print(f' - has dict: {hasattr(ep, "dict")}')
print(f' - slots: {getattr(ep.class, "slots", "NOT FOUND")}')
Test EndPoint base class
print(f'EndPoint.slots: {getattr(EndPoint, "slots", "NOT FOUND")}')
Test that resolve() method works (polymorphism)
for ep in endpoints:
try:
resolved = ep.resolve()
print(f'{ep.class.name}.resolve() -> {resolved}')
except Exception as e:
print(f'{ep.class.name}.resolve() failed: {e}')
print('\nInheritance tests completed successfully!')` (dns block)
This pull request was created as a result of the following prompt from Copilot chat.
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.