Implement lists:flatten/1 as a NIF
#1954
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pguyot
force-pushed
the
w44/lists-flatten-as-nif
branch
from
3c3efb6 to
005cf1e
Compare
Literal pools need to be within 1024 bytes because of limited range of ldr reg, [pc, #imm] instruction. A pool is emitted when mov_immediate is called and the pool reference is farther than a given threshold. Emitting the pool in such a condition requires a branch over the pool, and is therefore less efficient than the usual pool flush that happens when a branch is emitted (tail calls). This threshold was set to 900 but new test module bigint.beam wouldn't compile. This commit changes the threshold to 512 which works (the test actually requires 663). The optimal threshold value depends on the pattern usage of mov_immediate. If regular code hits the bug, we may need to change the heuristic and emit the pool based on other conditions. Signed-off-by: Paul Guyot <[email protected]>
Signed-off-by: Paul Guyot <[email protected]>
pguyot
force-pushed
the
w44/lists-flatten-as-nif
branch
from
005cf1e to
d623db2
Compare
bettio
reviewed
Nov 3, 2025
| ?ASSERT_MATCH(?MODULE:id([a, b]), [a, b]), | ||
| ok. | ||
|
|
||
| test_flatten() -> |
There was a problem hiding this comment.
I suggest adding a test for improper lists. Tests like:
[7 | {}][[] | [5 | 5]][[7 | 4], 2]
| if (term_is_nonempty_list(t)) { | ||
| term t_head = term_get_list_head(t); | ||
| term t_tail = term_get_list_tail(t); | ||
| if (term_is_nonempty_list(t_tail)) { |
There was a problem hiding this comment.
It looks good, but my guess is that it can be replaced with a slightly optimized alternative implementation that doesn't push all values, but only when there are nested lists.
This should optimize the trivial case (such as [1, 2, 3, 4]), by avoiding copying to the temp stack the entire list.
Basically we should push t_tail to the stack every time t_head is a term_is_nonempty_list.
In that case t_tail is used as a "continuation value" as soon as the current list is finished.
e.g.
flatten([[1, 2], 5, 6, 7]):
iteration 0:
- t is [[1, 2], 5, 6, 7]
- t_head is [1, 2]
- t_tail is [5, 6, 7]
- t_tail is pushed to the stack
- t is set to [1, 2]
...
when t is []:
- t is popped from the stack
iteration n:
- t is [5, 6, 7]
...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See #1953
These changes are made under both the "Apache 2.0" and the "GNU Lesser General
Public License 2.1 or later" license terms (dual license).
SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later