[WIP][Runtime]Pipeline Executor For Compute graph pipeline #7892

huajsj · 2021-04-20T21:50:36Z

Issue:
SOC hardware plarform have multiple types compute chipset like
GPU,FPGA,APU,RPU etc, there is a requirement that use these compute
unit in parallel to reach best performance.

Solution:
In these pipeline solution, we first split the compute graph into
a group of subgraph, then run these subgraph in a pipeline module
to make the GPU/FPGA/APU/RPU parallel running become possible.

Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.

tqchen · 2021-04-26T12:46:50Z

cc @comaniac @areusch @tmoreau89

huajsj · 2021-04-26T19:01:08Z

Thanks @tqchen for the follow up, I proposed a RFC(https://discuss.tvm.apache.org/t/rfc-compute-graph-pipeline-with-new-subgraph-executor/9839) to explain the motivation and solution architecture as reference.

comaniac

Reviewed the test for APIs and user interfaces, but it needs lots of changes so I'll stop here for now.

tests/python/relay/test_analysis_pipeline.py

huajsj · 2021-06-01T20:39:10Z

@comaniac @areusch @tmoreau89

comaniac

Alco cc @areusch @tmoreau89

cmake/config.cmake

python/tvm/contrib/pipeline_executor.py

comaniac · 2021-06-01T22:06:01Z

python/tvm/contrib/pipeline_executor.py

+        params : dict of str to NDArray
+           Additional arguments
+        """
+        if key is not None:


Why key and value are allowed to be None?

Didn't see the fix? Why we need if key is not None?

src/runtime/pipeline/pipeline_executor.cc

huajsj · 2021-06-17T20:44:34Z

@comaniac

comaniac · 2021-06-17T22:15:29Z

@comaniac

I'm busy with other tasks in recent days. Will try to take another look when I got a chance.

cmake/config.cmake

python/tvm/contrib/graph_executor.py

python/tvm/contrib/pipeline_executor.py

src/runtime/pipeline/pipeline_executor.h

src/runtime/pipeline/pipeline_executor.cc

src/runtime/pipeline/pipeline_executor.h

huajsj · 2021-07-07T19:01:24Z

@comaniac

huajsj · 2021-07-09T18:00:39Z

@comaniac, if you have time , could you help for a review. thanks.

masahi · 2021-09-03T07:13:16Z

src/runtime/pipeline/pipeline_data.h

+
+template <typename SLOT_TYPE = SLOT>
+void deleteQueue(squeue<SLOT_TYPE>* q) {
+  free(q);


masahi · 2021-09-03T07:16:22Z

src/runtime/pipeline/pipeline_struct.h

+#include <assert.h>
+#include <sched.h>
+#include <string.h>
+#include <sys/syscall.h>


Only compiles on linux

masahi · 2021-09-03T07:21:58Z

Thanks @huajsj. Please respect our coding style and standard good programming practices:

For C++ code: Variable names should be snake_case and function / class names should be CamelCase
Do not use using namespace without good reason
Do not introduce complicated typedef like typedef unordered_map<int, unordered_map<int, unordered_map<int, string>>> PIPELINE_CONF;. Introducing a class should be more readable.
Do not use weird type name like RUNTIME_PIPELINE_OUTPUT_CONF.

[Finding] the final output is same with constant value, seems like input data is 0, this is because the get input index have a '+1' operation, but the input index already start from 0. the means when doing setinput('x',..) it should get convert to setinput(0, ..), but the wrong logic is setinput(1,..).

comaniac · 2021-11-02T20:30:25Z

IIUC, this PR should be out-of-date? Should we close?

huajsj · 2021-11-05T06:17:33Z

@comaniac , yes we should close this PR, closed it now.

huajsj changed the title ~~Pipeline Compute Graph With New Subgraph Executor~~ [Runtime]Pipeline Compute Graph With New Subgraph Executor Apr 21, 2021

huajsj force-pushed the main branch from e02484f to 7c4dc14 Compare April 22, 2021 05:53

tqchen added the status: need review label Apr 26, 2021

huajsj changed the title ~~[Runtime]Pipeline Compute Graph With New Subgraph Executor~~ [WIP][Runtime]Pipeline Compute Graph With New Subgraph Executor Apr 28, 2021

huajsj force-pushed the main branch from 9f2918c to 7f0997e Compare May 1, 2021 06:46

huajsj changed the title ~~[WIP][Runtime]Pipeline Compute Graph With New Subgraph Executor~~ [Runtime]Pipeline Executor For Compute graph pipeline May 1, 2021

comaniac requested changes May 4, 2021

View reviewed changes

huajsj force-pushed the main branch from 5acbc8a to d77cfd4 Compare May 5, 2021 03:08

huajsj force-pushed the main branch from 222898e to 445488c Compare May 27, 2021 20:13

huajsj requested a review from comaniac June 1, 2021 18:42

comaniac requested changes Jun 1, 2021

View reviewed changes

huajsj force-pushed the main branch 4 times, most recently from 6061ae2 to 1c67a87 Compare June 16, 2021 21:52

huajsj requested a review from comaniac June 16, 2021 23:01

huajsj force-pushed the main branch from 1c67a87 to 5389f15 Compare June 17, 2021 04:44

comaniac requested changes Jun 23, 2021

View reviewed changes

huajsj force-pushed the main branch 2 times, most recently from 2947327 to 62e4d5f Compare June 30, 2021 06:17

huajsj requested a review from comaniac July 6, 2021 18:53

huajsj force-pushed the main branch from 7be31ae to 4bc78ab Compare July 13, 2021 01:49

huajsj mentioned this pull request Jul 29, 2021

[RFC][Tracking Issue] Pipeline Executor For Compute graph pipeline #8596

Closed

15 tasks

huajsj force-pushed the main branch from 4bc78ab to 29ecb2b Compare September 2, 2021 23:58

huajsj requested review from a team, ZihengJiang, areusch, jroesch, junrushao, kazum, liangfu, masahi, merrymercy, tmoreau89, tqchen, vinx13, yzhliu and zhiics as code owners September 2, 2021 23:58

masahi reviewed Sep 3, 2021

View reviewed changes

src/runtime/pipeline/pipeline_data.h

template <typename SLOT_TYPE = SLOT>

void deleteQueue(squeue<SLOT_TYPE>* q) {

free(q);

Copy link

Member

masahi Sep 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use delete

masahi reviewed Sep 3, 2021

View reviewed changes

change output start from 0

ad09daa

HungYangChang approved these changes Oct 1, 2021

View reviewed changes

huajsj added 6 commits October 1, 2021 18:34

Add network split logic into test.

47c5cc2

change moudle index into start 0.

b42de5a

fix mod_idx = 0 crash issue, but still logic wrong

25faf19

fix crash issue.

2e0c1c3

fix set param report error issue.

1ee5831

huajsj closed this Nov 5, 2021

huajsj mentioned this pull request Feb 1, 2022

[Runtime][ThreadPool]Refactor affinity function and support CPU affinity list setting. #9802

Merged

[WIP][Runtime]Pipeline Executor For Compute graph pipeline #7892

[WIP][Runtime]Pipeline Executor For Compute graph pipeline #7892

Uh oh!

Conversation

huajsj commented Apr 20, 2021

Uh oh!

tqchen commented Apr 26, 2021

Uh oh!

huajsj commented Apr 26, 2021

Uh oh!

comaniac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

huajsj commented Jun 1, 2021

Uh oh!

comaniac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

comaniac Jun 1, 2021

Choose a reason for hiding this comment

Uh oh!

huajsj Jun 4, 2021

Choose a reason for hiding this comment

Uh oh!

comaniac Jun 22, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

huajsj commented Jun 17, 2021

Uh oh!

comaniac commented Jun 17, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

huajsj commented Jul 7, 2021

Uh oh!

huajsj commented Jul 9, 2021

Uh oh!

masahi Sep 3, 2021

Choose a reason for hiding this comment

Uh oh!

masahi Sep 3, 2021

Choose a reason for hiding this comment

Uh oh!

masahi commented Sep 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

comaniac commented Nov 2, 2021

Uh oh!

huajsj commented Nov 5, 2021

Uh oh!

Reviewers

Assignees

Labels

masahi commented Sep 3, 2021 •

edited

Loading