Skip to content

Commit ef909df

Browse files
weberlotqchen
authored andcommitted
Implementation of uTVM (#3227)
* uTVM interfaces (#14) * some minor interface changes * implemented HostLowLevelDevice * added MicroDeviceAPI * implemented micro_common and added Python interfaces * current status, semi implemented micro session * added micro_common implementation and python interfaces (#18) * added micro_common implementation and python interfaces (#18) * current status, semi implemented * host test working * updated interfaces for MicroSession arguments allocation * make somewhat lint compatible * fix based on comments * added rounding macro * fix minor bug * improvements based on comments * Clean up `binutil.py` and make Python-3-compatible * Change argument allocation design * Address feedback and lint errors * Improve binutil tests * Simplify allocator (per @tqchen's suggestions) * Doc/style fixes * farts * mcgee * rodata section werks (and so does `test_runtime_micro_workspace.py`) * simple graph runtime werk * TEMP * ResNet works, yo * First round of cleanup * More cleanup * runs a dyson over the code * Another pass * Fix `make lint` issues * ready to pr... probably * final * Undo change * Fix rebase resolution * Minor fixes * Undo changes to C codegen tests * Add `obj_path` in `create_micro_lib` * TEMP * Address feedback * Add missing TODO * Partially address feedback * Fix headers * Switch to enum class for `SectionKind` * Add missing ASF header * Fix lint * Fix lint again * Fix lint * Kill lint warnings * Address feedback * Change Python interface to MicroTVM All interaction with the device is now through `Session` objects, which are used through Python's `with` blocks. * Reorder LowLevelDevice interface * Store shared ptr to session in all alloced objects * Move helper functions out of `tvm.micro` * Switch static char arr to vector * Improve general infra and code quality Does not yet address all of tqchen's feedback * Forgot a rename * Fix lint * Add ASF header * Fix lint * Partially address MarisaKirisame's feedback * Lint * Expose `MicroSession` as a node to Python * Revert to using `Session` constructor * Fix compiler error * (Maybe) fix CI error * Debugging * Remove * Quell lint * Switch to stack-based session contexts * Make uTVM less intrusive to host codegen And use SSA for operands of generated ternary operators * Inline UTVMArgs into UTVMTask struct * Remove `HostLowLevelDevice` header * Remove `BaseAddr` class * Address feedback * Add "utvm" prefix to global vars in runtime * Fix lint * Fix CI * Fix `test_binutil.py` * Fix submodules * Remove ResNet tests * Make `test_binutil.py` work with nose * Fix CI * I swear this actually fixes the binutil tests * lint * lint * Add fcompile-compatible cross-compile func * Add docs for uTVM runtime files * Move pointer patching into `MicroSession` * Fix lint * First attempt at unifying cross-compile APIs * Fix lint * Rename `cross_compile` back to `cc` * Address feedback * Remove commented code * Lint * Figure out failing function * Remove debugging code * Change "micro_dev" target to "micro" * Add checks in tests for whether uTVM is enabled * Add TODO for 32-bit support * Rename more "micro_dev" to "micro" * Undo rename We already have `tvm.micro` as a namespace. Can't have it as a method as well. * Fix failing CI Thanks to @tqchen for finding this bug. Emitting ternary operators for `min` and `max` causes concurrency bugs in CUDA, so we're moving the ternary op emissions from `CodeGenC` to `CodeGenCHost`. * Address feedback * Fix lint
1 parent 443d023 commit ef909df

37 files changed

+3321
-73
lines changed

CMakeLists.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ tvm_option(USE_RELAY_DEBUG "Building Relay in debug mode..." OFF)
3636
tvm_option(USE_SGX "Build with SGX" OFF)
3737
tvm_option(USE_RTTI "Build with RTTI" ON)
3838
tvm_option(USE_MSVC_MT "Build with MT" OFF)
39+
tvm_option(USE_MICRO "Build with Micro" OFF)
3940
tvm_option(INSTALL_DEV "Install compiler infrastructure" OFF)
4041
tvm_option(HIDE_PRIVATE_SYMBOLS "Compile with -fvisibility=hidden." OFF)
4142

@@ -206,6 +207,7 @@ include(cmake/modules/Metal.cmake)
206207
include(cmake/modules/ROCM.cmake)
207208
include(cmake/modules/SGX.cmake)
208209
include(cmake/modules/LLVM.cmake)
210+
include(cmake/modules/Micro.cmake)
209211
include(cmake/modules/ANTLR.cmake)
210212
include(cmake/modules/contrib/BLAS.cmake)
211213
include(cmake/modules/contrib/Random.cmake)

cmake/config.cmake

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,9 @@ set(USE_VULKAN OFF)
6262
# Whether enable OpenGL runtime
6363
set(USE_OPENGL OFF)
6464

65+
# Whether enable MicroTVM runtime
66+
set(USE_MICRO OFF)
67+
6568
# Whether to enable SGX runtime
6669
#
6770
# Possible values for USE_SGX:

cmake/modules/Micro.cmake

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
if(USE_MICRO)
19+
message(STATUS "Build with Micro support")
20+
file(GLOB RUNTIME_MICRO_SRCS src/runtime/micro/*.cc)
21+
list(APPEND RUNTIME_SRCS ${RUNTIME_MICRO_SRCS})
22+
endif(USE_MICRO)

include/tvm/runtime/c_runtime_api.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,7 @@ typedef enum {
8181
kDLAOCL = 5,
8282
kDLSDAccel = 6,
8383
kOpenGL = 11,
84+
kDLMicroDev = 13,
8485
// AddExtraTVMType which is not in DLPack here
8586
} TVMDeviceExtType;
8687

include/tvm/runtime/device_api.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -215,6 +215,7 @@ inline const char* DeviceName(int type) {
215215
case kDLROCM: return "rocm";
216216
case kOpenGL: return "opengl";
217217
case kDLExtDev: return "ext_dev";
218+
case kDLMicroDev: return "micro_dev";
218219
default: LOG(FATAL) << "unknown type =" << type; return "Unknown";
219220
}
220221
}

python/tvm/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@
4242

4343
from . import ndarray as nd
4444
from .ndarray import context, cpu, gpu, opencl, cl, vulkan, metal, mtl
45-
from .ndarray import vpi, rocm, opengl, ext_dev
45+
from .ndarray import vpi, rocm, opengl, ext_dev, micro_dev
4646

4747
from ._ffi.runtime_ctypes import TypeCode, TVMType
4848
from ._ffi.ndarray import TVMContext

python/tvm/_ffi/runtime_ctypes.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,7 @@ class TVMContext(ctypes.Structure):
143143
10: 'rocm',
144144
11: 'opengl',
145145
12: 'ext_dev',
146+
13: 'micro_dev',
146147
}
147148
STR2MASK = {
148149
'llvm': 1,
@@ -163,6 +164,7 @@ class TVMContext(ctypes.Structure):
163164
'rocm': 10,
164165
'opengl': 11,
165166
'ext_dev': 12,
167+
'micro_dev': 13,
166168
}
167169
def __init__(self, device_type, device_id):
168170
super(TVMContext, self).__init__()

python/tvm/contrib/binutil.py

Lines changed: 258 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,258 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
"""Utilities for binary file manipulation"""
19+
import os
20+
import subprocess
21+
from . import util
22+
from .._ffi.base import py_str
23+
from ..api import register_func
24+
25+
@register_func("tvm_callback_get_section_size")
26+
def tvm_callback_get_section_size(binary_path, section_name, toolchain_prefix):
27+
"""Finds size of the section in the binary.
28+
Assumes `size` shell command exists (typically works only on Linux machines)
29+
30+
Parameters
31+
----------
32+
binary_path : str
33+
path of the binary file
34+
35+
section_name : str
36+
name of section
37+
38+
toolchain_prefix : str
39+
prefix for binary names in target compiler toolchain
40+
41+
Returns
42+
-------
43+
size : integer
44+
size of the section in bytes
45+
"""
46+
if not os.path.isfile(binary_path):
47+
raise RuntimeError("no such file \"{}\"".format(binary_path))
48+
# We use the "-A" flag here to get the ".rodata" section's size, which is
49+
# not included by default.
50+
size_proc = subprocess.Popen(
51+
["{}size".format(toolchain_prefix), "-A", binary_path], stdout=subprocess.PIPE)
52+
(size_output, _) = size_proc.communicate()
53+
size_output = size_output.decode("utf-8")
54+
if size_proc.returncode != 0:
55+
msg = "error in finding section size:\n"
56+
msg += py_str(out)
57+
raise RuntimeError(msg)
58+
59+
# TODO(weberlo): Refactor this method and `*relocate_binary` so they are
60+
# both aware of [".bss", ".sbss", ".sdata"] being relocated to ".bss".
61+
section_mapping = {
62+
".text": [".text"],
63+
".rodata": [".rodata"],
64+
".data": [".data", ".sdata"],
65+
".bss": [".bss", ".sbss"],
66+
}
67+
sections_to_sum = section_mapping["." + section_name]
68+
section_size = 0
69+
# Skip the first two header lines in the `size` output.
70+
for line in size_output.split("\n")[2:]:
71+
tokens = list(filter(lambda s: len(s) != 0, line.split(" ")))
72+
if len(tokens) != 3:
73+
continue
74+
entry_name = tokens[0]
75+
entry_size = int(tokens[1])
76+
if entry_name in sections_to_sum:
77+
section_size += entry_size
78+
return section_size
79+
80+
81+
@register_func("tvm_callback_relocate_binary")
82+
def tvm_callback_relocate_binary(
83+
binary_path, text_addr, rodata_addr, data_addr, bss_addr, toolchain_prefix):
84+
"""Relocates sections in the binary to new addresses
85+
86+
Parameters
87+
----------
88+
binary_path : str
89+
path of the binary file
90+
91+
text_addr : str
92+
text section absolute address
93+
94+
rodata_addr : str
95+
rodata section absolute address
96+
97+
data_addr : str
98+
data section absolute address
99+
100+
bss_addr : str
101+
bss section absolute address
102+
103+
toolchain_prefix : str
104+
prefix for binary names in target compiler toolchain
105+
106+
Returns
107+
-------
108+
rel_bin : bytearray
109+
the relocated binary
110+
"""
111+
tmp_dir = util.tempdir()
112+
rel_obj_path = tmp_dir.relpath("relocated.o")
113+
ld_script_contents = ""
114+
# TODO(weberlo): There should be a better way to configure this for different archs.
115+
if "riscv" in toolchain_prefix:
116+
ld_script_contents += "OUTPUT_ARCH( \"riscv\" )\n\n"
117+
# TODO(weberlo): Generate the script in a more procedural manner.
118+
ld_script_contents += """
119+
SECTIONS
120+
{
121+
. = %s;
122+
. = ALIGN(8);
123+
.text :
124+
{
125+
*(.text)
126+
. = ALIGN(8);
127+
*(.text*)
128+
}
129+
. = %s;
130+
. = ALIGN(8);
131+
.rodata :
132+
{
133+
*(.rodata)
134+
. = ALIGN(8);
135+
*(.rodata*)
136+
}
137+
. = %s;
138+
. = ALIGN(8);
139+
.data :
140+
{
141+
*(.data)
142+
. = ALIGN(8);
143+
*(.data*)
144+
. = ALIGN(8);
145+
*(.sdata)
146+
}
147+
. = %s;
148+
. = ALIGN(8);
149+
.bss :
150+
{
151+
*(.bss)
152+
. = ALIGN(8);
153+
*(.bss*)
154+
. = ALIGN(8);
155+
*(.sbss)
156+
}
157+
}
158+
""" % (text_addr, rodata_addr, data_addr, bss_addr)
159+
rel_ld_script_path = tmp_dir.relpath("relocated.lds")
160+
with open(rel_ld_script_path, "w") as f:
161+
f.write(ld_script_contents)
162+
ld_proc = subprocess.Popen(["{}ld".format(toolchain_prefix), binary_path,
163+
"-T", rel_ld_script_path,
164+
"-o", rel_obj_path],
165+
stdout=subprocess.PIPE,
166+
stderr=subprocess.STDOUT)
167+
(out, _) = ld_proc.communicate()
168+
if ld_proc.returncode != 0:
169+
msg = "linking error using ld:\n"
170+
msg += py_str(out)
171+
raise RuntimeError(msg)
172+
with open(rel_obj_path, "rb") as f:
173+
rel_bin = bytearray(f.read())
174+
return rel_bin
175+
176+
177+
@register_func("tvm_callback_read_binary_section")
178+
def tvm_callback_read_binary_section(binary, section, toolchain_prefix):
179+
"""Returns the contents of the specified section in the binary byte array
180+
181+
Parameters
182+
----------
183+
binary : bytearray
184+
contents of the binary
185+
186+
section : str
187+
type of section
188+
189+
toolchain_prefix : str
190+
prefix for binary names in target compiler toolchain
191+
192+
Returns
193+
-------
194+
section_bin : bytearray
195+
contents of the read section
196+
"""
197+
tmp_dir = util.tempdir()
198+
tmp_bin = tmp_dir.relpath("temp.bin")
199+
tmp_section = tmp_dir.relpath("tmp_section.bin")
200+
with open(tmp_bin, "wb") as out_file:
201+
out_file.write(bytes(binary))
202+
objcopy_proc = subprocess.Popen(["{}objcopy".format(toolchain_prefix), "--dump-section",
203+
".{}={}".format(section, tmp_section),
204+
tmp_bin],
205+
stdout=subprocess.PIPE,
206+
stderr=subprocess.STDOUT)
207+
(out, _) = objcopy_proc.communicate()
208+
if objcopy_proc.returncode != 0:
209+
msg = "error in using objcopy:\n"
210+
msg += py_str(out)
211+
raise RuntimeError(msg)
212+
if os.path.isfile(tmp_section):
213+
# Get section content if it exists.
214+
with open(tmp_section, "rb") as f:
215+
section_bin = bytearray(f.read())
216+
else:
217+
# Return empty bytearray if the section does not exist.
218+
section_bin = bytearray("", "utf-8")
219+
return section_bin
220+
221+
222+
@register_func("tvm_callback_get_symbol_map")
223+
def tvm_callback_get_symbol_map(binary, toolchain_prefix):
224+
"""Obtains a map of symbols to addresses in the passed binary
225+
226+
Parameters
227+
----------
228+
binary : bytearray
229+
contents of the binary
230+
231+
toolchain_prefix : str
232+
prefix for binary names in target compiler toolchain
233+
234+
Returns
235+
-------
236+
map_str : str
237+
map of defined symbols to addresses, encoded as a series of
238+
alternating newline-separated keys and values
239+
"""
240+
tmp_dir = util.tempdir()
241+
tmp_obj = tmp_dir.relpath("tmp_obj.bin")
242+
with open(tmp_obj, "wb") as out_file:
243+
out_file.write(bytes(binary))
244+
nm_proc = subprocess.Popen(["{}nm".format(toolchain_prefix), "-C", "--defined-only", tmp_obj],
245+
stdout=subprocess.PIPE,
246+
stderr=subprocess.STDOUT)
247+
(nm_output, _) = nm_proc.communicate()
248+
if nm_proc.returncode != 0:
249+
msg = "error in using nm:\n"
250+
msg += py_str(nm_output)
251+
raise RuntimeError(msg)
252+
nm_output = nm_output.decode("utf8").splitlines()
253+
map_str = ""
254+
for line in nm_output:
255+
line = line.split()
256+
map_str += line[2] + "\n"
257+
map_str += line[0] + "\n"
258+
return map_str

0 commit comments

Comments
 (0)