Apigee Proxy Coverage

This is a tool to determine test case coverage for Apigee proxies.

Given a fully functional API proxy, the tool produces an instrumented proxy file as output. The instrumented proxy file should be deployed to Apigee Edge against which the test cases should be run. Upon completion, the execution data is retrieved by the tool from Apigee Edge to produce the coverage report.

The tool determines coverage of single API proxy under test.

Coverage in case of Apigee proxies is the percentage of policies executed with respect to the total number of policies. Unlike, programming languages where the practical acceptable coverage is around 80%, in case of Apigee proxies the coverage must be 100% - there are no policies that need not be tested or have a remote possibility of execution.

Usage

You can download the distribution using the following URL - https://github.com/apigee/apigee-proxy-coverage/releases/latest/download/apc.tar.gz

Pre-requisties

Apigee Edge / OPDK

The distribution provides shell scripts, a executable Java 17 jar file and python scripts - which makes bash, python, Java 17 as pre-requisites.

Apigee X / Hybrid

The distribution provides shell scripts, a executable Java 17 jar file and scripts - which makes bash, apigeecli, gcloud, curl, jq, Java 17 as pre-requisites.

Pick the appropriate distribution and follow the steps below

---

Steps: (Short Version)

1. Configure environment settings in config_vars
2. Run instr_prepare.sh
3. Run your test cases against the instrumented proxy deployed by previous script
4. Run instr_report.sh to obtain the coverage report.

---

Steps: (A bit more detail)

1. Enter your environment details in the file config_vars. The details of Apigee org, env, api proxy for coverage and the revision of the proxy to be used for instrumentation. Credentials to access the org is essential.
2. Run instr_prepare.sh. Following are steps performed by this script.
   • Download specified revision of proxy bundle from the org.
   • Instrument the downloaded bundle.
   • Deploy the instrumented bundle into the org and activate it.
   • Delete any instrumented data recorded against the proxy in the org.
3. Run all the tests against the deployed instrumented bundle. The instrument bundle has the same base path as the original bundle
4. Run instr_report.sh to obtain the coverage results. Following are the steps performed by this script.
   • Download the instrumented data recorded in the Key Value map in Apigee edge.
   • Generate the coverage report. The report has XML format - a summary.xml and one XML corresponding to every proxy and target endpoints are generated.
   • Custom stylesheets can be used convert the XMLs into desired report format.
   • A HTML version of the report is also generated by applying the default stylesheet.
   • Open the summary.html in the default browser.

Sample Report Screenshots

Summary

Policy Execution Drilldown

Coverage Numbers

Coverage is percentage of policies executed against the total number of policies that exists in the API proxy. With this definition it's possible to have overall coverage at 100%, while the flow level coverage be less than 100%. This is because all the policies would have been executed - but not on all paths.

It's best to achieve 100% flow level coverage in effect attain 100% overall coverage, than vice versa.

Limitations

All proxy endpoints are expected to have only one

• PreFlow tag
• PostFlow tag
• FaultRules tag

Apigee allows multiple of the above tags, though the behaviour in certain cases is un-documented and not supported for instrumentation.

The tool does not instrument the sharedflows that are called within the proxy

Internals

The tool has to determine the policies that executed in each flow to arrive at the final coverage number.

The tool instruments a given proxy using the following approach.

1. For each policy a new key-value map policy is added to record the execution.
2. The policy is added prior to the actual policy with the same condition as the one that exists on the actual.
3. The rationale is that if the key-value map policy executes so will the actual policy since the same condition is satisfied.
4. Further, its added prior to the actual policy to handle the scenario of fault policies in which case the policy following the fault will not be executed.
5. The generated policies have random unique names to avoid conflict. They log the information into a Key Value Map named instrument at the api proxy level.
6. After test execution, the tool uses the content of the key value map to reconstruct the execution information and calculate coverage.

Support

Issues filed on Github are not subject to service level agreements (SLAs) and responses should be assumed to be on an ad-hoc volunteer basis. The Apigee community board is recommended as for community support and is regularly checked by Apigee experts. Apigee customers should use [formal support channels](https://cloud.google.com/apigee/support for Apigee product related concerns.

• Post a question in Apigee community board
• Create an issue. Please specify if its for Apigee X/hybrid or Apigee Edge/OPDK

Disclaimer

This is not an officially supported Google product.