|
| 1 | +# Licensed to the Apache Software Foundation (ASF) under one |
| 2 | +# or more contributor license agreements. See the NOTICE file |
| 3 | +# distributed with this work for additional information |
| 4 | +# regarding copyright ownership. The ASF licenses this file |
| 5 | +# to you under the Apache License, Version 2.0 (the |
| 6 | +# "License"); you may not use this file except in compliance |
| 7 | +# with the License. You may obtain a copy of the License at |
| 8 | +# |
| 9 | +# http://www.apache.org/licenses/LICENSE-2.0 |
| 10 | +# |
| 11 | +# Unless required by applicable law or agreed to in writing, |
| 12 | +# software distributed under the License is distributed on an |
| 13 | +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| 14 | +# KIND, either express or implied. See the License for the |
| 15 | +# specific language governing permissions and limitations |
| 16 | +# under the License. |
| 17 | + |
| 18 | +""" |
| 19 | +.. _tutorial-micro-tvmc: |
| 20 | +
|
| 21 | +Executing a Tiny Model with TVMC Micro |
| 22 | +====================================== |
| 23 | +**Author**: `Mehrdad Hessar <https://github.com/mehrdadh>`_ |
| 24 | +
|
| 25 | +This tutorial explains how to compile a tiny model for a micro device, |
| 26 | +build a program on Zephyr platform to execute this model, flash the program |
| 27 | +and run the model all using `tvmc micro` command. |
| 28 | +""" |
| 29 | + |
| 30 | +###################################################################### |
| 31 | +# .. note:: |
| 32 | +# This tutorial is explaining using TVMC Mirco on Zephyr platform. You need |
| 33 | +# to install Zephyr dependencies before processing with this tutorial. Alternatively, |
| 34 | +# you can run this tutorial in one of the following ways which has Zephyr depencencies already installed. |
| 35 | +# |
| 36 | +# * Use `microTVM Reference Virtual Machines <https://tvm.apache.org/docs/how_to/work_with_microtvm/micro_reference_vm.html#sphx-glr-how-to-work-with-microtvm-micro-reference-vm-py>`_. |
| 37 | +# * Use QEMU docker image provided by TVM. Following these you will download and login to the docker image: |
| 38 | +# |
| 39 | +# .. code-block:: bash |
| 40 | +# |
| 41 | +# cd tvm |
| 42 | +# ./docker/bash.sh tlcpack/ci-qemu |
| 43 | +# |
| 44 | + |
| 45 | + |
| 46 | +############################################################ |
| 47 | +# Using TVMC Micro |
| 48 | +############################################################ |
| 49 | +# |
| 50 | +# TVMC is a command-line tool which is installed as a part of TVM Python packages. Accessing this |
| 51 | +# package varies based on your machine setup. In many cases, you can use the ``tvmc`` command directly. |
| 52 | +# Alternatively, if you have TVM as a Python module on your ``$PYTHONPATH``, you can access this |
| 53 | +# driver with ``python -m tvm.driver.tvmc`` command. This tutorial will use TVMC command as |
| 54 | +# ``tvmc`` for simplicity. |
| 55 | +# |
| 56 | +# To check if you have TVMC command installed on your machine, you can run: |
| 57 | +# |
| 58 | +# .. code-block:: bash |
| 59 | +# |
| 60 | +# tvmc --help |
| 61 | +# |
| 62 | +# To compile a model for microtvm we use ``tvmc compile`` subcommand. The output of this command |
| 63 | +# is used in next steps with ``tvmc micro`` subcommands. You can check the availability of TVMC Micro using: |
| 64 | +# |
| 65 | +# .. code-block:: bash |
| 66 | +# |
| 67 | +# tvmc micro --help |
| 68 | +# |
| 69 | +# The main tasks that you can perform using ``tvmc micro`` are ``create``, ``build`` and ``flash``. |
| 70 | +# To read about specific options under a givern subcommand, use |
| 71 | +# ``tvmc micro <subcommand> --help``. We will use each subcommand in this tutorial. |
| 72 | +# |
| 73 | + |
| 74 | +############################################################ |
| 75 | +# Obtain a Tiny Model |
| 76 | +############################################################ |
| 77 | +# |
| 78 | +# For this tutorial, we will use Magic Wand model from tflite micro. Magic Wand is a |
| 79 | +# Depthwise Convolution Layer model which recognizes gestures with an accelerometer. |
| 80 | +# |
| 81 | +# For this tutorial we will be using the model in tflite format. |
| 82 | +# |
| 83 | +# .. code-block:: bash |
| 84 | +# |
| 85 | +# wget https://github.com/tensorflow/tflite-micro/raw/main/tensorflow/lite/micro/examples/magic_wand/magic_wand.tflite |
| 86 | +# |
| 87 | + |
| 88 | +############################################################ |
| 89 | +# Compiling a TFLite model to a Model Library Format |
| 90 | +############################################################ |
| 91 | +# |
| 92 | +# Model Library Format (MLF) is an output format that TVM provides for micro targets. MLF is a tarball |
| 93 | +# containing a file for each piece of the TVM compiler output which can be used on micro targets outside |
| 94 | +# TVM environment. Read more about `Model Library Format <https://tvm.apache.org/docs//arch/model_library_format.html>`_. |
| 95 | +# |
| 96 | +# Here, we generate a MLF file for ``qemu_x86`` Zephyr board. To generate MLF output for the ``magic_wand`` tflite model: |
| 97 | +# |
| 98 | +# .. code-block:: bash |
| 99 | +# |
| 100 | +# tvmc compile magic_wand.tflite \ |
| 101 | +# --target='c -keys=cpu -link-params=0 -model=host' \ |
| 102 | +# --runtime=crt \ |
| 103 | +# --runtime-crt-system-lib 1 \ |
| 104 | +# --executor='graph' \ |
| 105 | +# --executor-graph-link-params 0 \ |
| 106 | +# --output model.tar \ |
| 107 | +# --output-format mlf \ |
| 108 | +# --pass-config tir.disable_vectorize=1 \ |
| 109 | +# --disabled-pass=AlterOpLayout |
| 110 | +# |
| 111 | +# This will generate a ``model.tar`` file which contains TVM compiler output files. To run this command for |
| 112 | +# a different Zephyr device, you need to update ``target``. For instance, for ``nrf5340dk_nrf5340_cpuapp`` board |
| 113 | +# the target is ``--target='c -keys=cpu -link-params=0 -model=nrf5340dk'``. |
| 114 | +# |
| 115 | + |
| 116 | + |
| 117 | +############################################################ |
| 118 | +# Create a Zephyr Project Using Model Library Format |
| 119 | +############################################################ |
| 120 | +# |
| 121 | +# To generate a Zephyr project we use TVM Micro subcommand ``create``. We pass the MLF format and the path |
| 122 | +# for the project to ``create`` subcommand along with project options. Project options for each |
| 123 | +# platform (Zephyr/Arduino) are defined in their Project API server file. To generate Zephyr project, run: |
| 124 | +# |
| 125 | +# .. code-block:: bash |
| 126 | +# |
| 127 | +# tvmc micro create \ |
| 128 | +# project \ |
| 129 | +# model.tar \ |
| 130 | +# zephyr \ |
| 131 | +# --project-option project_type=host_driven zephyr_board=qemu_x86 |
| 132 | +# |
| 133 | +# This will generate a ``Host-Driven`` Zephyr project for ``qemu_x86`` Zephyr board. In Host-Driven template project, |
| 134 | +# the Graph Executor will run on host and perform the model execution on Zephyr device by issuing commands to the |
| 135 | +# device using an RPC mechanism. Read more about `Host-Driven Execution <https://tvm.apache.org/docs/arch/microtvm_design.html#host-driven-execution>`_. |
| 136 | +# |
| 137 | +# To get more information about TVMC Micro ``create`` subcommand: |
| 138 | +# |
| 139 | +# .. code-block:: bash |
| 140 | +# |
| 141 | +# tvmc micro create --help |
| 142 | +# |
| 143 | + |
| 144 | +############################################################ |
| 145 | +# Build and Flash Zephyr Project Using TVMC Micro |
| 146 | +############################################################ |
| 147 | +# |
| 148 | +# Next step is to build the Zephyr project which includes TVM generated code for running the tiny model, Zephyr |
| 149 | +# template code to run a model in Host-Driven mode and TVM runtime source/header files. To build the project: |
| 150 | +# |
| 151 | +# .. code-block:: bash |
| 152 | +# |
| 153 | +# tvmc micro build \ |
| 154 | +# project \ |
| 155 | +# zephyr \ |
| 156 | +# --project-option zephyr_board=qemu_x86 |
| 157 | +# |
| 158 | +# This will build the project in ``project`` directory and generates binary files under ``project/build``. To build |
| 159 | +# Zephyr project for a different Zephyr board, change ``zephyr_board`` project option. |
| 160 | +# |
| 161 | +# Next, we flash the Zephyr binary file to Zephyr device. For ``qemu_x86`` Zephyr board this step does not |
| 162 | +# actually perform any action since QEMU will be used, however you need this step for physical hardware. |
| 163 | +# |
| 164 | +# .. code-block:: bash |
| 165 | +# |
| 166 | +# tvmc micro flash \ |
| 167 | +# project \ |
| 168 | +# zephyr \ |
| 169 | +# --project-option zephyr_board=qemu_x86 |
| 170 | +# |
| 171 | + |
| 172 | +############################################################ |
| 173 | +# Run Tiny Model on Micro Target |
| 174 | +############################################################ |
| 175 | +# |
| 176 | +# After flashing the device, the compiled model and TVM RPC server are programmed on the device. |
| 177 | +# The Zephyr board is waiting for host to open a communication channel. MicroTVM devices typicall communicate |
| 178 | +# using a serial communication (UART). To run the flashed model on the device using TVMC, we use ``tvmc run`` subcommand |
| 179 | +# and pass ``--device micro`` to specify the device type. This command will open a communication channel, set input |
| 180 | +# values using ``Graph Executor`` on host and run full model on the device. Then it gets output from the device. |
| 181 | +# |
| 182 | +# .. code-block:: bash |
| 183 | +# |
| 184 | +# tvmc run \ |
| 185 | +# --device micro \ |
| 186 | +# project \ |
| 187 | +# --project-option zephyr_board=qemu_x86 \ |
| 188 | +# --fill-mode ones |
| 189 | +# --print-top 4 |
| 190 | +# # Output: |
| 191 | +# # |
| 192 | +# # INFO:__main__:b'[100%] [QEMU] CPU: qemu32,+nx,+pae\n' |
| 193 | +# # remote: microTVM Zephyr runtime - running |
| 194 | +# # INFO:__main__:b'[100%] Built target run\n' |
| 195 | +# # [[3. 1. 2. 0. ] |
| 196 | +# # [0.47213247 0.41364592 0.07525456 0.03896701]] |
| 197 | +# |
| 198 | +# Specifically, this command sets the input of the model to all ones and shows the four values of the output with their indices. |
0 commit comments