feat: improve actor tool output

[MQ37]

closes https://github.com/apify/apify-mcp-server-internal/issues/147

Backstory

This PR rework how the MCP server handles the Actor tool output. Instead of dumping the whole dataset with filtered only the important fields which sometimes may cause LLM context or tool call limit overflow here we try to return feasible preview of the Actor output items and then provide a get-actor-output tool for output dataset items retrieval.

Details

The dataset items preview first tries to return the whole dataset with all items, if that does not fit into the 20k tokens limit (set to 20k to keep it conservative as some client put hard limit of 25k so there is some padding) then we include only important fields (fields, that are defined in some dataset view) and if that does not fit either then we try to truncate the items with important fields only one by one until we find number of important fields items that fit (in extreme case it can also retur no preview if the Actor has really bad and nested output format).

The get-actor-output tool is loaded whenever any tool that even indirectly interacts with Actors is present (call-actor, add-actor, Actor tool directly like apify-slash-rag-web-browser).

Changes

• Reworked and centralized the Actor tool output format. Now it returns metadata (runId, datasetId, total items count), then the output items schema, if possible some data preview and then instructions how to retrieve additional data or specific fields.
• Introduced new simple get-actor-output tools as I found the get-dataset-items complicated and wanted to keep this as original dataset centric advanced tool
• Centralized dataset items schema generation logic and removed the additionalProperties input from the get-dataset-schema as it seemed useless @jirispilka correct me if I am wrong
• Deleted the legacy remove-tool and helper-tool code
• Refactored all tool to use the central Ajv instance
• Centralized string comma-separated list parsing logic

[MichalKalita]

all await client.close() are not needed, because of afterEach on line 90.
AfterEach is safe, it's called if any test failed.

[MQ37]

I just put it there, just in case :D

[MichalKalita]

It's not removed 😄

[MQ37]

I did not remove it 😄

[MQ37]

I would keep that, it should not cause any issues, right?

[MichalKalita]

Manually closing is required only when more clients will be in a single test.

Every client will be closed by

afterEach(async () => {
            await client?.close();
            client = undefined;
        });

Even if the test failed, afterEach is always run, and the client will be closed and disconnected. Without this afterEach, each test should use try..finally to close client after.

It was a bug before. When one test failed, the client was not disconnected, and afterAll failed on timeout because the server was waiting for the client.

[jirispilka]

It works great — I played with it, and so far it’s flawless 🙂

My biggest concern is that get-actor-output shows up without users explicitly knowing it.

For example, when you configure tools, you end up with one extra tool as a kind of “bonus.” I think get-actor-output should go into the actors category. It makes sense there, typically when you call an Actor, you also need to retrieve its output.

What do you think?

[jirispilka]

I understand that logically, getActorOutput should be placed here BUT it might be really misleading for user. You will enable tools=actors and you will get an extra bonus of getActorOutput without any obvious reason. What about to add it into actors?

[MQ37]

Good catch! This should not be here, I would personally not add this tool to any category and kept it as "magical" tool that appears when you include any tool that can interact with Actors in any way (see README change that explains this). What do you think?

[jirispilka]

I think the fewer surprises, the better. When a user specifies something, we shouldn’t magically include something else. That feels like a big surprise and just asks for trouble — what if the user doesn’t want it at all? And how would they switch it off? There’s no way…

Instead, I’d add it to the actors category (alongside call-actor), and maybe also to other categories if needed.

But, t should also be included whenever an Actor is added 🤔, ahh

[MQ37]

I would personally keep this as a "magical" tool since user may not know when to add this tool and then might be frustrated when he only gets partial or no Actor results (we encountered user issues and reports like this already). I understand that now it seems more "magical" and is kind of a surprise but it makes the whole Actor calling and output retrieval work and this is crucial for the workflows. So I would not put this tool into any category and keep it hidden and auto-enabled if needed (based on the tool loader logic). I think there is no reason to Actually switch off this tool if you are working with Actors. Maybe there is, when you want to only call some Actor like send email and you do not care about the results, but that is miniory of the workflows and I would not complicate tool configuration for majorty only to satisfy this one use case where user might want to turn this tool off.