Skip to content
Merged
7 changes: 5 additions & 2 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -69,15 +69,18 @@ if ("${MBED_CONFIG_DEFINITIONS}" MATCHES "MBED_CONF_APP_REGRESSION_TEST=1")
platform_ns
tfm_ns_integration_test
tfm_qcbor
tfm_t_cose
)
# Firmware Update test supports Musca B1/M2354 only
# Firmware Update test supports Musca B1 and NU_M2354
if((${MBED_TARGET} STREQUAL ARM_MUSCA_B1) OR (${MBED_TARGET} STREQUAL NU_M2354))
list(APPEND TEST_LIBS
tfm_test_suite_fwu_ns
tfm_api_ns
)
endif()
# IRQ test only supports Musca B1
if(${MBED_TARGET} STREQUAL ARM_MUSCA_B1)
list(APPEND TEST_LIBS tfm_test_suite_irq)
endif()
elseif ("${MBED_CONFIG_DEFINITIONS}" MATCHES "MBED_CONF_APP_PSA_COMPLIANCE_TEST=1")
set(TEST_LIBS
val_nspe
Expand Down
25 changes: 14 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,17 +175,7 @@ python3 test_psa_target.py -h

## Expected test results

When you automate all tests, the Greentea test tool compares the test results with the logs in [`test/logs`](./test/logs) and prints a test report. *All test suites should pass*, except for the following suites that are currently **excluded** from the run:

* PSA Crypto suite: Some test cases are known to crash and reboot the target. This
causes the Greentea test framework to lose synchronization, and the residual data in the
memory prevents subsequent suites from running.

**Tip**: You can flash and run the PSA Crypto suite separately. Make sure
to build the Crypto suite manually with `wait-for-sync` set to 0 in
`mbed_app.json`, and power cycle the target before and after
the run to clear the memory. The total number of failures should match
`CRYPTO.log` in [`test/logs`](./test/logs)`/<your-target>`.
When you automate all tests, the Greentea test tool compares the test results with the logs in [`test/logs`](./test/logs) and prints a test report. *All test suites should pass or match the numbers of known failures in the logs.*

* M2354 hasn't supported PSA compliance test yet.

Expand Down Expand Up @@ -221,3 +211,16 @@ as follows
```
srec_cat tmp.bin -Binary -offset 0x00200000 -o musca_s1_ps_erase.hex -Intel
```

### Firmware Update test failures on M2354

The M2354 build configures TF-M to use a microSD card as the update staging area. Insert a microSD card into the slot on the board to allow the tests to pass.

Alternatively you can configure TF-M to use embedded flash as the update staging area with the following CMake variables:

```
set(NU_UPDATE_STAGE_SDH ON CACHE BOOL "Whether enable SDH as update staging area")
set(NU_UPDATE_STAGE_FLASH OFF CACHE BOOL "Whether enable embedded flash as update staging area")
```

The configuration variables must be passed in on the command line when building TF-M with CMake. The provided scripts in this repository don't have a mechanism for forwarding command line arguments to CMake, so to configure TF-M in this way you have to build TF-M using CMake commands.
2 changes: 1 addition & 1 deletion mbed-os.lib
Original file line number Diff line number Diff line change
@@ -1 +1 @@
https://github.com/ARMmbed/mbed-os/
https://github.com/rwalton-arm/mbed-os/#TF-Mv1.4.0
4 changes: 2 additions & 2 deletions psa_builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
"released-tfm": {
"trusted-firmware-m": [
"https://git.trustedfirmware.org/TF-M/trusted-firmware-m.git",
"TF-Mv1.3.0",
"TF-Mv1.4.0",
],
},
"latest-tfm": {
Expand All @@ -46,7 +46,7 @@
"nuvoton-tfm": {
"trusted-firmware-m": [
"https://github.com/OpenNuvoton/trusted-firmware-m",
"nuvoton_mbed_m2354_tfm-1.3",
"nuvoton_mbed_m2354_tfm-1.4",
],
},
}
Expand Down
2 changes: 1 addition & 1 deletion test/logs/ARM_MUSCA_B1/CRYPTO.log
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ Crypto Suite Report
TOTAL TESTS : [0-9]+
TOTAL PASSED : [0-9]+
TOTAL SIM ERROR : 0
TOTAL FAILED : 18
TOTAL FAILED : 23
TOTAL SKIPPED : 0
21 changes: 11 additions & 10 deletions test/logs/ARM_MUSCA_B1/REGRESSION.log
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
Non-secure test suites summary
Test suite 'PSA protected storage NS interface tests \(TFM_PS_TEST_1XXX\)' has .* PASSED
Test suite 'PSA internal trusted storage NS interface tests \(TFM_ITS_TEST_1XXX\)' has .* PASSED
Test suite 'Crypto non-secure interface test \(TFM_CRYPTO_TEST_6XXX\)' has .* PASSED
Test suite 'Platform Service Non-Secure interface tests\(TFM_PLATFORM_TEST_2XXX\)' has .* PASSED
Test suite 'Initial Attestation Service non-secure interface tests\(TFM_ATTEST_TEST_2XXX\)' has .* PASSED
Test suite 'QCBOR regression test\(TFM_QCBOR_TEST_7XXX\)' has .* PASSED
Test suite 'T_COSE regression test\(TFM_T_COSE_TEST_8XXX\)' has .* PASSED
Test suite 'PSA firmware update NS interface tests \(TFM_FWU_TEST_1xxx\)' has .* PASSED
Test suite 'Core non-secure positive tests \(TFM_CORE_TEST_1XXX\)' has .* PASSED
Test suite 'IPC non-secure interface test \(TFM_IPC_TEST_1XXX\)' has .* PASSED
Test suite 'PSA protected storage NS interface tests \(TFM_NS_PS_TEST_1XXX\)' has.*PASSED
Test suite 'PSA internal trusted storage NS interface tests \(TFM_NS_ITS_TEST_1XXX\)' has.*PASSED
Test suite 'Crypto non-secure interface test \(TFM_NS_CRYPTO_TEST_1XXX\)' has.*PASSED
Test suite 'Platform Service Non-Secure interface tests\(TFM_NS_PLATFORM_TEST_1XXX\)' has.*PASSED
Test suite 'Initial Attestation Service non-secure interface tests\(TFM_NS_ATTEST_TEST_1XXX\)' has.*PASSED
Test suite 'QCBOR regression test\(TFM_NS_QCBOR_TEST_1XXX\)' has.*PASSED
Test suite 'T_COSE regression test\(TFM_NS_T_COSE_TEST_1XXX\)' has.*PASSED
Test suite 'PSA firmware update NS interface tests \(TFM_NS_FWU_TEST_1xxx\)' has.*PASSED
Test suite 'Core non-secure positive tests \(TFM_NS_CORE_TEST_1XXX\)' has.*PASSED
Test suite 'IPC non-secure interface test \(TFM_NS_IPC_TEST_1XXX\)' has.*PASSED
Test suite 'TFM IRQ Test \(TFM_IRQ_TEST_1xxx\)' has.*PASSED
End of Non-secure test suites
2 changes: 1 addition & 1 deletion test/logs/ARM_MUSCA_S1/CRYPTO.log
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ Crypto Suite Report
TOTAL TESTS : [0-9]+
TOTAL PASSED : [0-9]+
TOTAL SIM ERROR : 0
TOTAL FAILED : 17
TOTAL FAILED : 23
TOTAL SKIPPED : 0
18 changes: 9 additions & 9 deletions test/logs/ARM_MUSCA_S1/REGRESSION.log
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
Non-secure test suites summary
Test suite 'PSA protected storage NS interface tests \(TFM_PS_TEST_1XXX\)' has .* PASSED
Test suite 'PSA internal trusted storage NS interface tests \(TFM_ITS_TEST_1XXX\)' has .* PASSED
Test suite 'Crypto non-secure interface test \(TFM_CRYPTO_TEST_6XXX\)' has .* PASSED
Test suite 'Platform Service Non-Secure interface tests\(TFM_PLATFORM_TEST_2XXX\)' has .* PASSED
Test suite 'Initial Attestation Service non-secure interface tests\(TFM_ATTEST_TEST_2XXX\)' has .* PASSED
Test suite 'QCBOR regression test\(TFM_QCBOR_TEST_7XXX\)' has .* PASSED
Test suite 'T_COSE regression test\(TFM_T_COSE_TEST_8XXX\)' has .* PASSED
Test suite 'Core non-secure positive tests \(TFM_CORE_TEST_1XXX\)' has .* PASSED
Test suite 'IPC non-secure interface test \(TFM_IPC_TEST_1XXX\)' has .* PASSED
Test suite 'PSA protected storage NS interface tests \(TFM_NS_PS_TEST_1XXX\)' has.*PASSED
Test suite 'PSA internal trusted storage NS interface tests \(TFM_NS_ITS_TEST_1XXX\)' has.*PASSED
Test suite 'Crypto non-secure interface test \(TFM_NS_CRYPTO_TEST_1XXX\)' has.*PASSED
Test suite 'Platform Service Non-Secure interface tests\(TFM_NS_PLATFORM_TEST_1XXX\)' has.*PASSED
Test suite 'Initial Attestation Service non-secure interface tests\(TFM_NS_ATTEST_TEST_1XXX\)' has.*PASSED
Test suite 'QCBOR regression test\(TFM_NS_QCBOR_TEST_1XXX\)' has.*PASSED
Test suite 'T_COSE regression test\(TFM_NS_T_COSE_TEST_1XXX\)' has.*PASSED
Test suite 'Core non-secure positive tests \(TFM_NS_CORE_TEST_1XXX\)' has.*PASSED
Test suite 'IPC non-secure interface test \(TFM_NS_IPC_TEST_1XXX\)' has.*PASSED
End of Non-secure test suites
20 changes: 10 additions & 10 deletions test/logs/NU_M2354/REGRESSION.log
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
Non-secure test suites summary
Test suite 'PSA protected storage NS interface tests \(TFM_PS_TEST_1XXX\)' has .* PASSED
Test suite 'PSA internal trusted storage NS interface tests \(TFM_ITS_TEST_1XXX\)' has .* PASSED
Test suite 'Crypto non-secure interface test \(TFM_CRYPTO_TEST_6XXX\)' has .* PASSED
Test suite 'Platform Service Non-Secure interface tests\(TFM_PLATFORM_TEST_2XXX\)' has .* PASSED
Test suite 'Initial Attestation Service non-secure interface tests\(TFM_ATTEST_TEST_2XXX\)' has .* PASSED
Test suite 'QCBOR regression test\(TFM_QCBOR_TEST_7XXX\)' has .* PASSED
Test suite 'T_COSE regression test\(TFM_T_COSE_TEST_8XXX\)' has .* PASSED
Test suite 'PSA firmware update NS interface tests \(TFM_FWU_TEST_1xxx\)' has .* PASSED
Test suite 'Core non-secure positive tests \(TFM_CORE_TEST_1XXX\)' has .* PASSED
Test suite 'IPC non-secure interface test \(TFM_IPC_TEST_1XXX\)' has .* PASSED
Test suite 'PSA protected storage NS interface tests \(TFM_NS_PS_TEST_1XXX\)' has.*PASSED
Test suite 'PSA internal trusted storage NS interface tests \(TFM_NS_ITS_TEST_1XXX\)' has.*PASSED
Test suite 'Crypto non-secure interface test \(TFM_NS_CRYPTO_TEST_1XXX\)' has.*PASSED
Test suite 'Platform Service Non-Secure interface tests\(TFM_NS_PLATFORM_TEST_1XXX\)' has.*PASSED
Test suite 'Initial Attestation Service non-secure interface tests\(TFM_NS_ATTEST_TEST_1XXX\)' has.*PASSED
Test suite 'QCBOR regression test\(TFM_NS_QCBOR_TEST_1XXX\)' has.*PASSED
Test suite 'T_COSE regression test\(TFM_NS_T_COSE_TEST_1XXX\)' has.*PASSED
Test suite 'PSA firmware update NS interface tests \(TFM_NS_FWU_TEST_1xxx\)' has.*PASSED
Test suite 'Core non-secure positive tests \(TFM_NS_CORE_TEST_1XXX\)' has.*PASSED
Test suite 'IPC non-secure interface test \(TFM_NS_IPC_TEST_1XXX\)' has.*PASSED
End of Non-secure test suites
13 changes: 3 additions & 10 deletions test_psa_target.py
Original file line number Diff line number Diff line change
Expand Up @@ -326,16 +326,9 @@ def _build_compliance_test(args, test_spec):
_build_mbed_os(args)
binary_name = _erase_flash_storage(args, suite)

# Issue: https://github.com/ARM-software/psa-arch-tests/issues/252
# The Crypto suite is known to crash and reset the target during runs.
# This causes the Greentea test framework to lose synchronization, and
# messes up the memory and prevents subsequent suites from running.
# The PSA tests currently provide no option to skip known failures.
# Users can still run the Crypto suite manually without automation.
if suite != "CRYPTO":
test_spec["builds"][test_group]["tests"][
suite.lower()
] = _get_test_spec(args, suite, binary_name)
test_spec["builds"][test_group]["tests"][
suite.lower()
] = _get_test_spec(args, suite, binary_name)


def _get_parser():
Expand Down
54 changes: 41 additions & 13 deletions tfm_ns_import.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,18 @@
{
"src": "install/image_signing/layout_files/signing_layout_s.o",
"dst": "targets/TARGET_ARM_SSG/TARGET_MUSCA_S1/partition/signing_layout_s.c"
},
{
"src": "lib/ext/tfm_test_repo-src/app/tfm_ns_interface.c",
"dst": "targets/TARGET_ARM_SSG/TARGET_MUSCA_S1/tfm_ns_interface.c"
},
{
"src": "../platform/ext/target/arm/musca_s1/partition/flash_layout.h",
"dst": "targets/TARGET_ARM_SSG/TARGET_MUSCA_S1/partition/flash_layout.h"
},
{
"src": "../platform/ext/target/arm/musca_s1/partition/region_defs.h",
"dst": "targets/TARGET_ARM_SSG/TARGET_MUSCA_S1/partition/region_defs.h"
}
],
"ARM_MUSCA_B1": [
Expand All @@ -38,9 +50,25 @@
{
"src": "install/image_signing/layout_files/signing_layout_s.o",
"dst": "targets/TARGET_ARM_SSG/TARGET_MUSCA_B1/partition/signing_layout_s.c"
},
{
"src": "lib/ext/tfm_test_repo-src/app/tfm_ns_interface.c",
"dst": "targets/TARGET_ARM_SSG/TARGET_MUSCA_B1/tfm_ns_interface.c"
},
{
"src": "../platform/ext/target/arm/musca_b1/sse_200/partition/flash_layout.h",
"dst": "targets/TARGET_ARM_SSG/TARGET_MUSCA_B1/partition/flash_layout.h"
},
{
"src": "../platform/ext/target/arm/musca_b1/sse_200/partition/region_defs.h",
"dst": "targets/TARGET_ARM_SSG/TARGET_MUSCA_B1/partition/region_defs.h"
}
],
"NU_M2354": [
{
"src": "install/interface/src/tfm_firmware_update_ipc_api.c",
"dst": "platform/FEATURE_EXPERIMENTAL_API/FEATURE_PSA/TARGET_TFM/TARGET_TFM_LATEST/src/tfm_firmware_update_ipc_api.c"
},
{
"src": "../platform/ext/target/nuvoton/m2354/partition/flash_layout.h",
"dst": "targets/TARGET_NUVOTON/TARGET_M2354/TARGET_TFM/TARGET_NU_M2354/COMPONENT_TFM_S_FW/partition/flash_layout.h"
Expand Down Expand Up @@ -130,10 +158,6 @@
"src": "install/interface/include",
"dst": "platform/FEATURE_EXPERIMENTAL_API/FEATURE_PSA/TARGET_TFM/TARGET_TFM_LATEST/include"
},
{
"src": "../interface/include/os_wrapper",
"dst": "platform/FEATURE_EXPERIMENTAL_API/FEATURE_PSA/TARGET_TFM/TARGET_TFM_LATEST/include/os_wrapper"
},
{
"src": "install/interface/include/psa",
"dst": "platform/FEATURE_EXPERIMENTAL_API/FEATURE_PSA/TARGET_TFM/TARGET_TFM_LATEST/include/psa"
Expand All @@ -145,17 +169,21 @@
{
"src": "lib/ext/tfm_test_repo-src/app/os_wrapper_cmsis_rtos_v2.c",
"dst": "platform/FEATURE_EXPERIMENTAL_API/FEATURE_PSA/TARGET_TFM/TARGET_TFM_LATEST/src/os_wrapper_cmsis_rtos_v2.c"
},
{
"src": "lib/ext/tfm_test_repo-src/ns_interface/os_wrapper",
"dst": "platform/FEATURE_EXPERIMENTAL_API/FEATURE_PSA/TARGET_TFM/TARGET_TFM_LATEST/include/os_wrapper"
},
{
"src": "../interface/include/tfm_psa_call_param.h",
"dst": "platform/FEATURE_EXPERIMENTAL_API/FEATURE_PSA/TARGET_TFM/TARGET_TFM_LATEST/include/tfm_psa_call_param.h"
}
],
"v8-m": [
{
"src": "install/interface/src/tfm_psa_ns_api.c",
"dst": "platform/FEATURE_EXPERIMENTAL_API/FEATURE_PSA/TARGET_TFM/TARGET_TFM_LATEST/TARGET_TFM_V8M/src/tfm_psa_ns_api.c"
},
{
"src": "install/interface/src/tfm_ns_interface.c",
"dst": "platform/FEATURE_EXPERIMENTAL_API/FEATURE_PSA/TARGET_TFM/TARGET_TFM_LATEST/TARGET_TFM_V8M/src/tfm_ns_interface.c"
}
],
"dualcpu": [
{
Expand Down Expand Up @@ -221,10 +249,6 @@
"src": "lib/ext/qcbor/libtfm_qcbor_test.a",
"dst": "test/lib"
},
{
"src": "lib/ext/t_cose/libtfm_t_cose.a",
"dst": "test/lib"
},
{
"src": "lib/ext/t_cose/libtfm_t_cose_test.a",
"dst": "test/lib"
Expand All @@ -246,13 +270,17 @@
"dst": "test/lib"
},
{
"src": "test/suites/fwu/libtfm_test_suite_fwu_ns.a",
"src": "test/suites/fwu/mcuboot/libtfm_test_suite_fwu_ns.a",
"dst": "test/lib"
},
{
"src": "test/suites/ipc/libtfm_test_suite_ipc_ns.a",
"dst": "test/lib"
},
{
"src": "test/suites/irq/libtfm_test_suite_irq.a",
"dst": "test/lib"
},
{
"src": "test/suites/its/libtfm_test_suite_its_ns.a",
"dst": "test/lib"
Expand Down