CAT in Tapscript (BIP-347)

[arminsabouri]

CAT in Tapscript

This PR provides the necessary code to enable the opcode OP_CAT in Tapscript as specified in BIP-347: OP_CAT in Tapscript and BIN-2024-0001,

Important: This PR does not include miner activation functionality. This means that merging this PR into bitcoin-core will not make OP_CAT functional in Bitcoin.
If this PR is merged it is not a signal of community consensus around activating CAT nor be read as a portent about the activation process or timeline.
The PR is not a stand-in for consensus around such decisions.

This PR includes:

• An implementation of the Tapscript opcode OP_CAT along with the flags SCRIPT_VERIFY_OP_CAT and SCRIPT_VERIFY_DISCOURAGE_OP_CAT.
• Integration and unit tests which ensure that our CAT implementation works as expected. This includes ensuring that CAT never uses more that 520 bytes of stack memory. A full description of what these tests cover is given below.
• An improvement to src/test/script_tests.cpp JSON script format enabling the rapid creation of new Tapscript unit tests.

Activation on Bitcoin-inquisition (signet)

Bitcoin-inquisition PR 37 which contained our implementation of OP_CAT was merged into bitcoin-inquisition (signet) on Apr 25 2024, released as bitcoin-inquisition release 25.2 on Apr 26 2024. and activated in bitcoin-inquisition Apr 30 2024. The bitcoin-inquisition PR was reviewed in the PR review club (transcript of discussion). Since Apr 30 2024 there have been many OP_CAT transactions created and spent on signet.

The code merged into bitcoin-inquisition differs from this PR, as we have removed the consensus logic which was used to activate it on signet.

OP_CAT Tapscript Implementation

We implement OP_CAT as a new Tapscript op code by redefining the opcode OP_SUCCESS126 (126 in decimal and 0x7e in hexadecimal). This is the same opcode value used by the original OP_CAT.

When evaluated, the OP_CAT instruction:

1. Pops the top two values off the stack,
2. concatenates the popped values together in stack order,
3. and then pushes the concatenated value on the top of the stack.

OP_CAT fails if there are fewer than two values on the stack or if a concatenated value would have a combined size greater than the maximum script element size of 520 bytes.

See BIP 347 for a deeper description.

Errors thrown

If evaluated OP_CAT can throw the following errors:

• If at time of evaluation the stack has fewer than two elements we throw the error: SCRIPT_ERR_INVALID_STACK_OPERATION
• If at time of evaluation the top two stack elements have a combined size greater than MAX_SCRIPT_ELEMENT_SIZE (520 bytes) we throw the error: SCRIPT_ERR_PUSH_SIZE.

Script verification flags

While this PR does not contain any miner signaling and activation logic and can not activate OP_CAT, it does contain two flags which future activation logic could set to control activation of OP_CAT.

• SCRIPT_VERIFY_OP_CAT IF a bitcoin node has this set to true, then it treat OP_CAT enabled for Tapscript. That is, OP_SUCCESS126 will be redefining to OP_CAT in Tapscript. If this was set to true at the consensus level this would cause a soft fork.

SCRIPT_VERIFY_DISCOURAGE_OP_CAT When set to true, a node receiving any Tapscript transaction containing the opcode OP_CAT or OP_SUCCESS126 will reject the transaction throwing the error SCRIPT_ERR_DISCOURAGE_OP_CAT but not banning the node which relayed the transaction. This prevents nodes from relaying transactions with OP_CAT. This is equivalent to the behavior of SCRIPT_ERR_DISCOURAGE_OP_CAT = true when SCRIPT_VERIFY_OP_CAT = false as OP_CAT is an OP_SUCCESS (OP_SUCCESS126).

This is how these two flags are intended to be used to ensure a smooth soft fork.

Stage	SCRIPT_VERIFY OP_CAT	SCRIPT_VERIFY DISCOURAGE_OP_CAT	Status
1.Default	False	True	This does not represent any change in behavior of the bitcoin-core node.
2.Upgrading	True	True	OP_CAT is activating.
3.Upgraded	True	False	It is clear that OP_CAT has been activated and the network has been upgraded.

The flag SCRIPT_ERR_DISCOURAGE_OP_CAT provides a window of time for the network to fully activate, before nodes will relay or accept transactions containing OP_CAT in their mem pools.

Tests

This PR contains a suite of script tests to ensure that OP_CAT functions as expected. In the JSON script tests (script_tests.json), we test:

• That if OP_CAT is not activated that there are no changes to bitcoin consensus.
• That regardless of the activation or non-activation of OP_CAT in Tapscript, pre-Tapscript scripts, i.e. Bitcoin scripts, have no changes in behavior.
• That OP_CAT when evaluated throws the expected error when there are less than two elements on the stack.
• That OP_CAT when evaluated in a variety of circumstances and edge cases, successfully concatenates elements of the stack. This includes:
  • multiple calls to OP_CAT in a row,
  • evaluating OP_CAT inside of a IF conditional,
  • evaluating OP_CAT on zero size stack elements, random and large stack elements,
  • evaluating OP_CAT on values being moved to and from the alt stack,
  • and checking that when evaluated OP_CAT concatenates the elements in the expected order.

All of these tests are designed to cover the happy path of OP_CAT, the various errors which OP_CAT can throw and all the corner cases between those two outcomes.

Additionally we include three tests outside of the JSON script tests.

• cat_simple and cat_empty_stack are designed to test OP_CAT outside of the JSON serialization regime. Ensure that we catch bugs that we might miss in the JSON script tests due to a bug introduced at the JSON serialization layer.
• cat_dup_test enumerates all stack element sizes from 1 to 522 bytes and then enumerates up to 10 repetitions of OP\_DUP OP\_CAT. It then tests if the stack element would exceed 520 bytes and if so did OP_CAT throw the error SCRIPT_ERR_PUSH_SIZE. This allows us to be certain that OP_CAT will not introduce any OP\_DUP OP\_CAT memory exhaustion attacks.

Better Tapscript tests in JSON script tests

While writing these JSON script tests (script_tests.json) we ran into the following problem. The JSON script tests are simple and easy to write for pre-Tapscript scripts, but adding or changing a Tapscript test requires substantial work per test.
Consider the following pre-tapscript test:

["'aa' 'bb'", "CAT 0x4c 0x02 0xaabb EQUAL", "P2SH,STRICTENC", "DISABLED_OPCODE", "CAT disabled"]

whereas a Tapscript test for the same script (annotated with comments for better readability) would look like:

[
    [
        "aa",
        "bb",
        "7e4c02aabb87", // output script
        "c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d", // control block
        0.00000001
    ],
    "",
    "0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99", // Tapscript output
    "P2SH,WITNESS,TAPROOT",
    "OK",
    "TAPSCRIPT CATs aa and bb together and checks if EQUAL to aabb"
]

Computing the Tapscript output, such as 0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99, requires writing custom code and running it for each test. The same is true for the Tapscript control block, such as c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d. If a test is changed or updated new outputs and control blocks must be computed. The complexity of doing this is likely the reason that no one has added any Tapscript tests to JSON script tests until this PR.

In this PR we address this issue by adding the following improvements to JSON script tests:

• Adding simple macros ("#SCRIPT# and #CONTROLBLOCK#) that allow the script test parser to automatically generate and inject a valid Tapscript output and control block to be computed automatically from the JSON script.
• Allowing Tapscript scripts to use the human readable strings like pre-script scripts by marking the location of the script in the witness stack using #SCRIPT#. This transforms the unreadable script 7e4c02aabb87 into #SCRIPT# CAT 0x4c 0x02 0xaabb EQUAL.

This results in the following JSON script test which is far easier to write and easier to read.

[
    [
        "aa",
        "bb",
        "#SCRIPT# CAT",
        "#CONTROLBLOCK#",
        0.00000001
    ],
    "",
    "0x51 0x20 #TAPROOTOUTPUT#",
    "P2SH,WITNESS,TAPROOT,OP_CAT",
    "OK",
    "TAPSCRIPT Test of OP_CAT flag by calling CAT on two elements. TAPSCRIPT_OP_CAT flag is set so CAT is executed."
],

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/29247.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #32998 (Bump SCRIPT_VERIFY flags to 64 bit by ajtowns)
• #32453 ([Policy] Discourage Unsigned Annexes by JeremyRubin)
• #31989 (BIP-119 (OP_CHECKTEMPLATEVERIFY) (regtest only) by jamesob)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

[DrahtBot]

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

_{Debug: https://github.com/bitcoin/bitcoin/runs/20473822722}

[arminsabouri]

Drafting until I get the inquisition PR approved and I can get the builds passing.

[moonsettler]

Some of us have been playing with the idea of neutered CAT: instead of the MAX_SCRIPT_ELEMENT_SIZE (520 bytes) the maximum output size would be 80 bytes.

Rationale:
It explicitly disables open ended second order effects by removing the ability to assemble a SIGHASH or CTV template on the stack for more detailed introspection than intended. If such introspection behavior is desired in the future it can be explicitly enabled by specialized and more efficient opcodes.

Combined with CSFS it can be used as a signed datacarrier, for which the 'standard' limit is 80 bytes, it also has to be smaller than 84 bytes which is required for building CTV templates on the stack, and larger than 64/65/72 bytes which are respectively needed for:

• Merkle inclusion: 2x 20/32 byte hashes
• LN-symmetry (LNhance): 2x 32 byte hashes
• Separate sig: 64/72 bytes (0-conf bonds, staking contracts)

Could be a livable compromise between the conservatives that want to preserve certain characteristics of bitcoin and the prometheans who want to give the developers more practical and useful tools to build with.

[bigspider]

Rationale: It explicitly disables open ended second order effects by removing the ability to assemble a SIGHASH or CTV template on the stack for more detailed introspection than intended. If such introspection behavior is desired in the future it can be explicitly enabled by specialized and more efficient opcodes.

Script is already expressive enough to (awkwardly, expensively) compute arbitrary SHA256 hashes on the stack, except that the result would be broken in smaller pieces of at most 4 bytes. With CAT, those can be concatenated to get a single 32-byte result.

Therefore, neutering CATs does not achieve the desired result of preventing the CHECKSIG tricks, unless you limit the length of the result to less than 32 bytes - which would also neuter most of the utility of the opcode.

[DrahtBot]

🚧 At least one of the CI tasks failed.
_{Debug: https://github.com/bitcoin/bitcoin/runs/29308305455}

Hints

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being
  incompatible with the current code in the target branch). If so, make sure to rebase on the latest
  commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the
  affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

[Co1nB3e]

What about SCRIPT_VERIFY OP_CAT = False and SCRIPT_VERIFY DISCOURAGE_OP_CAT = False?
This combination is not described in the "soft fork" table. Is it part of the tests?

[EthanHeilman]

@AlexSQY That's an interesting question.
Current Bitcoin behavior is OP_CAT = False and DISCOURAGE_OP_CAT = False. I don't believe we would ever want to set DISCOURAGE_OP_CAT = False while OP_CAT = False since a disabled opcode should be discouraged.

[Co1nB3e]

@EthanHeilman , thanks for the feedback.
I understand this combination should be discouraged.
My concern, perhaps not justified, was about this negative case not being documented/tested, thus wondering about a potential edge case.
Found out those tapscript test cases but I do not know if "not set" is exactly same as "set to false" (all those variables set to false by default?):
https://github.com/bitcoin/bitcoin/pull/29247/files#diff-ef0159635ee0fee8864cd9570a04337cabb63e9c49a9e96d7b746194ecaa427d

PS: DISCOURAGE_OP_SUCCESS also introduces more combinations one with the others (2 tested as part of the actual script_tests.json, nominal cases only?)

[EthanHeilman]

@AlexSQY I see your point. Let me see if there is anything that can be done to ensure it is never the case that OP_CAT = False && DISCOURAGE_OP_CAT = False since that should never happen outside of tests.

The behavior is what you see in that test is the expected behavior for DISCOURAGE_OP_CAT = False and OP_CAT = False,
since OP_CAT = False the CAT opcode is an OP_SUCCESS so it returns success (OK).

@0xBEEFCAF3 What do you think?

[arminsabouri]

Let me see if there is anything that can be done to ensure it is never the case that OP_CAT = False && DISCOURAGE_OP_CAT = False since that should never happen outside of tests.

By default, the SCRIPT_VERIFY_DISCOURAGE_OP_CAT bit is set, and we do not expose this as a configurable mempool policy. The only practical way for node operators to unset this bit (SCRIPT_VERIFY_DISCOURAGE_OP_CAT = False) is by compiling a custom binary.

That said, @AlexSQY, you're right -- this edge case deserves documentation. If I recall correctly, it’s being tested here, where neither the discourage bit nor the script verify bit is set. In that case, both are false, and 0x7e just behaves as OP_SUCCESS.

[ajtowns]

Current Bitcoin behavior is OP_CAT = False and DISCOURAGE_OP_CAT = False.

DISCOURAGE_OP_SUCCESS in current bitcoin covers the behaviour of DISCOURAGE_OP_CAT.

OP_CAT=false and DISCOURAGE_OP_CAT=false are used for validating blocks prior to activation of OP_CAT, so this isn't an edge case as far as I can see? OP_CAT=true and DISCOURAGE_OP_CAT=true should be an edge case, but it doesn't make sense of course: "disallow OP_CAT, but if it were allowed, which it's not, enforce its rules".

I think the table should look more like:

Phase	Mode	SV_OP_CAT	SV_DISCOURAGE_OP_CAT
Pre-activation	Transaction relay	❌ (false)	✔️ (true)
Pre-activation	Block validation	❌ (false)	❌ (false)
Post-activation	Transaction relay	✔️ (true)	❌ (false)
Post-activation	Block validation	✔️ (true)	❌ (false)

[DrahtBot]

🐙 This pull request conflicts with the target branch and needs rebase.