[Eng]: Smoke test improvements

[isra-fel]

Description

Reliability

Sometimes smoke test may fail to a 503 or timeout of an Azure API. We may consider adding retry for every test case to improve reliability.

PSGallery is also a main fact that smoke test fails.

Coverage

We are now testing PowerShell 6.2; 7.0 and 7.2. We should consider adding 7.1 and removing 6.2.

Performace

Smoke test pipeline clones the whole azure-powershell repo, which costs 4~5 minutes, but only a few files are actually used. We could investigate how to clone only necessary files to make it faster.

[dingmeng-xue]

In addition, we need to add more cmdlets into smoke test which module is GAed or depending on data plane. Such as AzAD cmdlets.

[msJinLei]

@LucasYao93 what's the status of the issue？

[LucasYao93]

@LucasYao93 what's the status of the issue？
Not finished. Need to discuss with YeMing.

[isra-fel]

About the performance part, since smoke test only depends on a few files instead of the whole big repo, it would save a lot of time to just clone the needed files.

Here's how to disable cloning the repo: https://docs.microsoft.com/en-us/azure/devops/pipelines/repos/github?view=azure-devops&tabs=classic#dont-sync-sources

Here's how to clone only selective directories. The following script only takes ~10 seconds!!

git clone --depth=1 --filter=blob:none --sparse https://github.com/Azure/azure-powershell.git
cd .\azure-powershell\
git sparse-checkout set .azure-pipelines
git sparse-checkout add tools/Test/SmokeTest
git sparse-checkout add (more)

Source: https://stackoverflow.com/a/52269934/2593588