Skip to content

Commit 6c889a0

Browse files
committed
--wip--
1 parent c287a03 commit 6c889a0

File tree

2 files changed

+1
-5
lines changed

2 files changed

+1
-5
lines changed

docs/architecture/distributed_runtime.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ For example, the deployment configuration `examples/llm/configs/disagg.yaml` hav
3434
- `Processor`: When a new request arrives, `Processor` applies the chat template and perform the tokenization. Then, it route the request to the `VllmWorker`.
3535
- `VllmWorker` and `PrefillWorker`: Perform the actual decode and prefill computation.
3636

37-
Since the four workers are deployed in different processes, each of them have their own `DistributedRuntime`. Within their own `DistributedRuntime`, they all have their own `Namespace`s named `dynamo`. Then, under their own `dynamo` namespace, they have their own `Component`s named `Frontend/Processor/VllmWorker/PrefillWorker`. Lastly, for the `Endpoint`, `Frontend` has no `Endpoints`, `Processor` and `VllmWorker` each has a `generate` endpoint, and `PrefillWorker` has a placeholder `mock` endpoint. Their `DistributedRuntime`s and `Namespace`s are set in the `@service` decorators in `examples/llm/components/<frontend/processor/worker/prefill_worker>.py`. Their `Component`s are set by their name in `/deploy/dynamo/sdk/src/dynamo/sdk/cli/serve_dynamo.py`. Their `Endpoint`s are set by the `@endpoint` decorators in `examples/llm/components/<frontend/processor/worker/prefill_worker>.py`.
37+
Since the four workers are deployed in different processes, each of them have their own `DistributedRuntime`. Within their own `DistributedRuntime`, they all have their own `Namespace`s named `dynamo`. Then, under their own `dynamo` namespace, they have their own `Component`s named `Frontend/Processor/VllmWorker/PrefillWorker`. Lastly, for the `Endpoint`, `Frontend` has no `Endpoints`, `Processor` and `VllmWorker` each has a `generate` endpoint, and `PrefillWorker` has a placeholder `mock` endpoint.
3838

3939
## Initialization
4040

docs/dynamo_glossary.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,6 @@
2424
**Dynamo Cloud** - A Kubernetes platform providing managed deployment experience for Dynamo inference graphs.
2525

2626
## E
27-
**@endpoint** - A Python decorator used to define service endpoints within a Dynamo component.
28-
2927
**Endpoint** - A specific network-accessible API within a Dynamo component, such as `generate` or `load_metrics`.
3028

3129
## F
@@ -70,8 +68,6 @@
7068
**RDMA (Remote Direct Memory Access)** - Technology that allows direct memory access between distributed systems, used for efficient KV cache transfers.
7169

7270
## S
73-
**@service** - Python decorator used to define a Dynamo service class.
74-
7571
**SGLang** - Fast LLM inference framework with native embedding support and RadixAttention.
7672

7773
## T

0 commit comments

Comments
 (0)