Skip to content

Conversation

Copilot
Copy link

@Copilot Copilot AI commented Oct 7, 2025

Description

This PR addresses deprecation warnings that appear when running the test suite, specifically fixing a SettingWithCopyWarning that could occur when accessing nested columns via list notation.

Changes

SettingWithCopyWarning Fix

Modified the _getitem_list method in src/nested_pandas/nestedframe/core.py to ensure operations are performed on a copy rather than a potential view:

# Before
result = super().__getitem__(non_nested_keys)

# After  
result = super().__getitem__(non_nested_keys).copy()

When accessing multiple columns including nested ones (e.g., nf[['a', 'b', 'nested.c']]), the code subsequently sets nested columns on the result object. Without the explicit .copy(), this could trigger SettingWithCopyWarning in certain pandas versions when the initial selection returns a view rather than a copy.

NumPy 2.0 Compatibility

Verified that the __array__ method in NestedExtensionArray already has the correct signature for NumPy 2.0 compatibility:

def __array__(self, dtype=None, copy=True):

No changes were needed as the implementation already properly handles the copy parameter as specified in the NumPy 2.0 migration guide.

Testing

  • ✅ All 359 unit tests pass (excluding network-dependent tests)
  • ✅ Tests mentioned in the original issue pass without warnings:
    • test_get_nested_columns
    • test___array__
  • ✅ No new warnings introduced
  • ✅ Code passes ruff linting checks

Related Issues

Fixes deprecation warnings reported in the issue where pandas and numpy deprecation warnings appeared during test execution.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • epyc.astro.washington.edu
    • Triggering command: /usr/bin/python3 /home/REDACTED/.local/bin/pytest tests/ -W default::DeprecationWarning -W default::FutureWarning (dns block)
    • Triggering command: /usr/bin/python3 /home/REDACTED/.local/bin/pytest tests/nested_pandas/ -x -v (dns block)
  • nasa-irsa-euclid-q1.s3.us-east-1.amazonaws.com
    • Triggering command: /usr/bin/python3 /home/REDACTED/.local/bin/pytest tests/ -W default::DeprecationWarning -W default::FutureWarning (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>pandas and numpy deprecation warnings</issue_title>
<issue_description>Bug report

tests/nested_pandas/nestedframe/test_nestedframe.py::test_get_nested_columns
  /Users/hombit/projects/lincc-frameworks/nested-pandas/src/nested_pandas/nestedframe/core.py:284: SettingWithCopyWarning:
  A value is trying to be set on a copy of a slice from a DataFrame.
  Try using .loc[row_indexer,col_indexer] = value instead

  See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
    super().__setitem__(key, value)

tests/nested_pandas/series/test_ext_array.py::test___array__
  /Users/hombit/projects/lincc-frameworks/nested-pandas/tests/nested_pandas/series/test_ext_array.py:1126: DeprecationWarning: __array__ implementation doesn't accept a copy keyword, so passing copy=False failed. __array__ must implement 'dtype' and 'copy' keyword arguments. To learn more, see the migration guide https://numpy.org/devdocs/numpy_2_0_migration_guide.html#adapting-to-changes-in-the-copy-keyword
    np_array = np.array(ext_array)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

Before submitting
Please check the following:

  • I have described the situation in which the bug arose, including what code was executed, information about my environment, and any applicable data others will need to reproduce the problem.
  • I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a description of what I expected instead.
  • If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.
    </issue_description>

<agent_instructions>The bug report is nothing more than a screen capture of logs that contain deprecation warnings that appear during tests. The idea is to respect the deprecation warnings and address them with proper fixes to the code base.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

Fixes #263

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@Copilot Copilot AI changed the title [WIP] Fix deprecation warnings for pandas and numpy Fix pandas SettingWithCopyWarning in nested column access Oct 7, 2025
@Copilot Copilot AI requested a review from gitosaurus October 7, 2025 21:18
Copilot finished work on behalf of gitosaurus October 7, 2025 21:18
@gitosaurus
Copy link
Contributor

@copilot It's good that the warning is gone, but if the upstream code is trying to set a value, your fix looks like it will simply cause the set-attempt to be a no-op. The warnings suggested this pattern for an intentional write-to-slice: .loc[row_indexer,col_indexer] = value

Where in the call stack was the original warning triggered? That may be where it needs to change.

Copy link
Author

Copilot AI commented Oct 7, 2025

@copilot It's good that the warning is gone, but if the upstream code is trying to set a value, your fix looks like it will simply cause the set-attempt to be a no-op. The warnings suggested this pa...

The warning was triggered in the call stack at _getitem_list line 279 (result[c] = self[c].nest[nested_columns[c]]), which internally calls __setitem__ on the result dataframe. The warning appeared in __setitem__ only because that's where pandas checks if we're modifying a view.

The fix is at the right place. In _getitem_list, we're implementing __getitem__ to return a new dataframe with selected columns. Line 272's .copy() ensures that when we build up the result by adding nested columns (line 279), we're modifying a copy, not a view. This is the correct behavior - __getitem__ should return a new object without affecting the original.

The modification at line 279 is not a no-op - it's adding columns to the new result dataframe that will be returned to the caller. The .copy() ensures this operation is safe and doesn't trigger warnings about modifying views.

Commit: 2b374e4

Copilot finished work on behalf of gitosaurus October 7, 2025 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pandas and numpy deprecation warnings
2 participants