Add miopen install rpm script #26

jataylo · 2023-03-09T10:09:13Z

Companion PR to https://github.com/ROCmSoftwarePlatform/rocAutomation/pull/114

Adds new common/install_rocm_miopen_rpm.sh file which will install a given MIOpen rpm package from a given source.

export MIOPEN_RPM_SOURCE=https://repo.radeon.com/rocm/yum/5.4/main/miopen-hip-2.19.0.50400-72.el7.x86_64.rpm

build_wheel.sh
In build_wheel.sh we have deployed an approach related to the MIOpen install approach from upstream https://github.com/pytorch/builder/blob/main/common/install_miopen.sh#L134

wget the ${MIOPEN_RPM_SOURCE} and throw an error if the download fails

+ docker exec -t manylinux-rocm-py3.8 bash -c 'source /env && bash /builder/common/install_rocm_miopen_rpm.sh'
Running on Linux 89640942d125 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux at Wed Mar  8 20:15:25 UTC 2023
+ EXTRACTDIR_ROOT=/extract_miopen_rpm
+ mkdir -p /extract_miopen_rpm
+ echo 'Creating temporary directory for rpm download...'
Creating temporary directory for rpm download...
+ wget -P /extract_miopen_rpm https://repo.radeon.com/rocm/yum/5.4/main/miopen-hip-2.19.0.50400-72.el7.x86_64.rpm
--2023-03-08 20:15:25--  https://repo.radeon.com/rocm/yum/5.4/main/miopen-hip-2.19.0.50400-72.el7.x86_64.rpm
Resolving repo.radeon.com (repo.radeon.com)... 13.82.220.49
Connecting to repo.radeon.com (repo.radeon.com)|13.82.220.49|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 83647440 (80M) [application/x-redhat-package-manager]
Saving to: ‘/extract_miopen_rpm/miopen-hip-2.19.0.50400-72.el7.x86_64.rpm’


 0% [                                       ] 0           --.-K/s              
 0% [                                       ] 188,133      823KB/s             
 1% [                                       ] 1,416,933   3.02MB/s             
 3% [>                                      ] 3,317,477   4.72MB/s             
 6% [=>                                     ] 5,218,021   5.57MB/s             
 8% [==>                                    ] 7,118,565   6.08MB/s             
10% [===>                                   ] 9,035,493   6.43MB/s             
13% [====>                                  ] 10,952,421  6.62MB/s             
15% [=====>                                 ] 12,869,349  6.81MB/s             
17% [=====>                                 ] 14,802,661  6.97MB/s             
20% [======>                                ] 16,735,973  7.09MB/s             
22% [=======>                               ] 18,685,669  7.20MB/s             
24% [========>                              ] 20,618,981  7.29MB/s             
27% [=========>                             ] 22,585,061  7.37MB/s             
29% [==========>                            ] 24,534,757  7.43MB/s  eta 8s     
31% [===========>                           ] 26,500,837  8.07MB/s  eta 8s     
34% [============>                          ] 28,466,917  8.11MB/s  eta 8s     
36% [=============>                         ] 30,449,381  8.12MB/s  eta 8s     
38% [==============>                        ] 32,431,845  8.23MB/s  eta 8s     
41% [===============>                       ] 34,430,693  8.23MB/s  eta 6s     
43% [===============>                       ] 36,413,157  8.31MB/s  eta 6s     
45% [================>                      ] 37,953,253  8.34MB/s  eta 6s     
47% [=================>                     ] 39,821,029  8.35MB/s  eta 6s     
50% [==================>                    ] 41,852,645  8.44MB/s  eta 6s     
52% [===================>                   ] 43,884,261  8.51MB/s  eta 5s     
54% [====================>                  ] 45,424,357  8.44MB/s  eta 5s     
56% [=====================>                 ] 47,455,973  8.50MB/s  eta 5s     
59% [======================>                ] 49,602,277  8.67MB/s  eta 5s     
61% [======================>                ] 51,191,525  8.60MB/s  eta 5s     
63% [=======================>               ] 53,354,213  8.70MB/s  eta 4s     
66% [========================>              ] 55,566,053  8.86MB/s  eta 4s     
69% [=========================>             ] 57,794,277  8.99MB/s  eta 4s     
71% [==========================>            ] 59,514,597  8.93MB/s  eta 4s     
73% [===========================>           ] 61,873,893  9.12MB/s  eta 4s     
76% [============================>          ] 64,265,957  9.26MB/s  eta 2s     
79% [=============================>         ] 66,428,645  9.36MB/s  eta 2s     
81% [==============================>        ] 68,525,797  9.55MB/s  eta 2s     
84% [================================>      ] 71,081,701  9.75MB/s  eta 2s     
88% [=================================>     ] 73,719,525  9.94MB/s  eta 2s     
91% [==================================>    ] 76,185,317  10.2MB/s  eta 1s     
93% [===================================>   ] 78,405,349  10.2MB/s  eta 1s     
97% [====================================>  ] 81,239,781  10.5MB/s  eta 1s     
100%[======================================>] 83,647,440  10.7MB/s   in 8.9s   

2023-03-08 20:15:35 (8.92 MB/s) - ‘/extract_miopen_rpm/miopen-hip-2.19.0.50400-72.el7.x86_64.rpm’ saved [83647440/83647440]

+ echo 'MIOpen package download complete...'
MIOpen package download complete...

Extract the rpm package with rpm2cpio

+ cd /extract_miopen_rpm
++ ls miopen-hip-2.19.0.50400-72.el7.x86_64.rpm
+ miopen_rpm=miopen-hip-2.19.0.50400-72.el7.x86_64.rpm
+ rpm2cpio miopen-hip-2.19.0.50400-72.el7.x86_64.rpm
+ cpio -idmv
./opt
./opt/rocm-5.4.0
./opt/rocm-5.4.0/bin
./opt/rocm-5.4.0/bin/MIOpenDriver
./opt/rocm-5.4.0/bin/install_precompiled_kernels.sh
./opt/rocm-5.4.0/lib
./opt/rocm-5.4.0/lib/libMIOpen.so.1
./opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400
./opt/rocm-5.4.0/share
./opt/rocm-5.4.0/share/doc
./opt/rocm-5.4.0/share/doc/miopen-hip
./opt/rocm-5.4.0/share/doc/miopen-hip/LICENSE.txt
./opt/rocm-5.4.0/share/miopen
./opt/rocm-5.4.0/share/miopen/db
./opt/rocm-5.4.0/share/miopen/db/gfx1030_36.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx1030_36.db
./opt/rocm-5.4.0/share/miopen/db/gfx803_36.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx803_36.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx803_64.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx803_64.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx900_56.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx900_56.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx900_56.db
./opt/rocm-5.4.0/share/miopen/db/gfx900_64.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx900_64.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx906_60.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx906_60.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx906_60.db
./opt/rocm-5.4.0/share/miopen/db/gfx906_64.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx906_64.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx90878.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx90878.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx90878.db
./opt/rocm-5.4.0/share/miopen/db/gfx90a68.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx90a68.db
./opt/rocm-5.4.0/share/miopen/db/gfx90a6e.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx90a6e.db
1894145 blocks

Move the extracted libMIOpen.so.1.0* file to the target libMIOpen.so.1.0* location

++ ls opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400
+ source_file=opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400
++ ls /opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404
+ dest_file=/opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404
+ '[' -e opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400 ']'
+ '[' -e /opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404 ']'
+ echo 'Source .so: opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400'
Source .so: opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400
+ echo 'Dest .so: /opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404'
Dest .so: /opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404
+ cp opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400 /opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404
+ echo 'libMIOpen so file from RPM copied to existing MIOpen install...'
libMIOpen so file from RPM copied to existing MIOpen install...

Removes the temporary directory

+ rm -rf /extract_miopen_rpm
+ echo 'Removed temporary directory..

Example CI log:
Build MIOpen RPM source test

jithunnair-amd · 2023-03-22T06:03:59Z

common/install_rocm_miopen_rpm.sh

+rpm2cpio ${miopen_rpm} | cpio -idmv 
+
+# Copy libMIOpen.so.1 over existing
+source_file=$(ls opt/rocm-*/lib/libMIOpen.so.1.0*)


May not be an immediate issue, but the hardcoding of the MIOpen .so version bugs me :)

Add miopen install rpm script

b23275a

jataylo requested a review from jithunnair-amd March 9, 2023 10:09

jithunnair-amd approved these changes Mar 10, 2023

View reviewed changes

jithunnair-amd merged commit 82ccc1c into ROCm:main Mar 10, 2023

jithunnair-amd reviewed Mar 22, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add miopen install rpm script #26

Add miopen install rpm script #26

Uh oh!

jataylo commented Mar 9, 2023

Uh oh!

jithunnair-amd Mar 22, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add miopen install rpm script #26

Add miopen install rpm script #26

Uh oh!

Conversation

jataylo commented Mar 9, 2023

Uh oh!

jithunnair-amd Mar 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jithunnair-amd Mar 22, 2023 •

edited

Loading