Agentset - Partition API

This is the partition API used by the Agentset Platform.

Tech Stack

• Modal – deployments
• Unstructured – document parsing
• LlamaIndex chunking
• FastAPI – API

Deployment

1. Install dependencies:

uv sync

2. Create your modal API token (Manage Workspaces -> API Tokens)

3. Link modal cli to your account (copy the commands from the modal dashboard and run them with uv run):

uv run modal token set --token-id ak-xxx --token-secret as-xxx --profile=example

uv run modal profile activate example

4. Create secrets (read .env.example for more info about the variables):

uv run modal secret create --force partitioner-secrets QSTASH_URL=https://qstash.io QSTASH_TOKEN=xxx REDIS_HOST=xxx REDIS_PORT=xxx REDIS_PASSWORD=xxx AGENTSET_API_KEY=xxx

5. Deploy the app:

uv run modal deploy main.py

6. Done 🎉
   

Now take the URL from the modal dashboard PARTITION_API_URL and the AGENTSET_API_KEY you specified above, and add them to the Agentset Platform as PARTITION_API_URL and PARTITION_API_KEY, respectively.

They should look like this:

PARTITION_API_URL=https://example.modal.run/ingest
PARTITION_API_KEY=xxx

License

This project is open-source under the MIT License. You can find it here.