Skip to content

[Vulkan] Vulkan runtime reimplementation (stream approach) #3849

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 13, 2019
Merged

[Vulkan] Vulkan runtime reimplementation (stream approach) #3849

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions apps/android_rpc/app/src/main/jni/tvm_runtime.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,7 @@
#endif

#ifdef TVM_VULKAN_RUNTIME
#include "../src/runtime/vulkan/vulkan_device_api.cc"
#include "../src/runtime/vulkan/vulkan_module.cc"
#include "../src/runtime/vulkan/vulkan.cc"
#endif

#ifdef USE_SORT
Expand Down
26 changes: 23 additions & 3 deletions cmake/modules/Vulkan.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,14 @@
# Be compatible with older version of CMake
find_vulkan(${USE_VULKAN})

# Extra Vulkan runtime options, exposed for advanced users.
tvm_option(USE_VULKAN_IMMEDIATE_MODE "Use Vulkan Immediate mode
(KHR_push_descriptor extension)" ON IF USE_VULKAN)
tvm_option(USE_VULKAN_DEDICATED_ALLOCATION "Use Vulkan dedicated allocations" ON
IF USE_VULKAN)
tvm_option(USE_VULKAN_VALIDATION "Enable Vulkan API validation layers" OFF
IF USE_VULKAN)

if(Vulkan_FOUND)
# always set the includedir
# avoid global retrigger of cmake
Expand All @@ -28,12 +36,24 @@ if(USE_VULKAN)
if(NOT Vulkan_FOUND)
message(FATAL_ERROR "Cannot find Vulkan, USE_VULKAN=" ${USE_VULKAN})
endif()
message(STATUS "Build with VULKAN support")
file(GLOB RUNTIME_VULKAN_SRCS src/runtime/vulkan/*.cc)
message(STATUS "Build with Vulkan support")
file(GLOB RUNTIME_VULKAN_SRCS src/runtime/vulkan/vulkan.cc)
file(GLOB COMPILER_VULKAN_SRCS src/codegen/spirv/*.cc)
list(APPEND RUNTIME_SRCS ${RUNTIME_VULKAN_SRCS})
list(APPEND COMPILER_SRCS ${COMPILER_VULKAN_SRCS})

list(APPEND TVM_LINKER_LIBS ${Vulkan_SPIRV_TOOLS_LIBRARY})
list(APPEND TVM_RUNTIME_LINKER_LIBS ${Vulkan_LIBRARY})

if(USE_VULKAN_IMMEDIATE_MODE)
message(STATUS "Build with Vulkan immediate mode")
add_definitions(-DUSE_VULKAN_IMMEDIATE_MODE=1)
endif()
if(USE_VULKAN_DEDICATED_ALLOCATION)
message(STATUS "Build with Vulkan dedicated allocation")
add_definitions(-DUSE_VULKAN_DEDICATED_ALLOCATION=1)
endif()
if(USE_VULKAN_VALIDATION)
message(STATUS "Build with Vulkan API validation")
add_definitions(-DUSE_VULKAN_VALIDATION=1)
endif()
endif(USE_VULKAN)
2 changes: 2 additions & 0 deletions src/codegen/spirv/build_vulkan.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@

#include "codegen_spirv.h"
#include "../build_common.h"

#include "../../runtime/vulkan/vulkan_shader.h"
#include "../../runtime/vulkan/vulkan_module.h"

namespace tvm {
Expand Down
5 changes: 4 additions & 1 deletion src/codegen/spirv/ir_builder.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,10 @@ namespace spirv {
void IRBuilder::InitHeader() {
CHECK_EQ(header_.size(), 0U);
header_.push_back(spv::MagicNumber);
header_.push_back(spv::Version);
// Use SPIR-V v1.0. This needs to be kept in sync (or at least behind)
// `VkApplicationInfo.apiVersion` in `vulkan.cc` to ensure Vulkan API
// validation passes.
header_.push_back(0x10000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to hard-code the version like this? Is it possible to use spv::Version both here and in vulkcan2.cc and check that its >= v1.0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that the semantically correct thing to do is:

// generator: set to 0, unknown
header_.push_back(0U);
// Bound: set during Finalize
Expand Down
48 changes: 48 additions & 0 deletions src/runtime/vulkan/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<!--- Licensed to the Apache Software Foundation (ASF) under one -->
<!--- or more contributor license agreements. See the NOTICE file -->
<!--- distributed with this work for additional information -->
<!--- regarding copyright ownership. The ASF licenses this file -->
<!--- to you under the Apache License, Version 2.0 (the -->
<!--- "License"); you may not use this file except in compliance -->
<!--- with the License. You may obtain a copy of the License at -->

<!--- http://www.apache.org/licenses/LICENSE-2.0 -->

<!--- Unless required by applicable law or agreed to in writing, -->
<!--- software distributed under the License is distributed on an -->
<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
<!--- KIND, either express or implied. See the License for the -->
<!--- specific language governing permissions and limitations -->
<!--- under the License. -->


## Components

### VulkanDeviceAPI

Implements the TVM DeviceAPI interface. Owns the core Vulkan datastructures. Is
responsible for initializing the Vulkan instance and devices, querying for
possible extensions.

### VulkanThreadEntry

Thread-local state for the Vulkan runtime. Maintains a staging buffer (for
copies), and a VulkanStream per device.

### VulkanWrappedFunc

Responsible for launching computation kernels. Responsible for obtaining a
VulkanPipeline instance (from the VulkanModuleNode), and launches the kernel
(via immediate or deferred mode) on the active VulkanStream instance.

## Stream execution in the Vulkan programming model.

The natural model for TVM DeviceAPI implementation and runtime follows the CUDA
API model. That is, we launch "kernels" onto a (implicit or explicit) "stream"
(which execute asynchronously with respect to the host, but ordered with respect
to the stream), and explicitly synchronize the stream with respect to the host.
We simulate this behaviour in the Vulkan model by maintaining a thread-local
`vkCommandBuffer` instance, and queueing up (or eagerly executing, depending on
the availability of the `VK_KHR_push_descriptor` extension). When we synchronize
the stream, we end the command buffer recording, submit it to the device queue,
and wait on the corresponding fence.
Loading