Skip to content
Last updated

respect

Introduction

Use this command to execute API tests described in an Arazzo description.

Important

The respect command supports Arazzo 1.0.1 descriptions. Of the features added in Arazzo 1.1.0, it supports only the parameters field of success and failure actions. When an action references a workflow with workflowId, respect passes the action's parameters to that workflow as inputs.

Usage

npx @redocly/cli@latest respect <your-test-file | multiple files | files bash query> [-w | --workflow] [-s | --skip] [-v | --verbose] [-i | --input] [-S | --server] [-H | --har-output] [-J | --json-output] [--max-steps] [--max-fetch-timeout] [--execution-timeout] [--severity] [--no-secrets-masking]

Options

Option Type Description

-w, --workflow

[string]

Workflow names from the test file to run. For example, the following command runs "first-flow" and "second-flow" workflows from the test-file.yaml Arazzo description:

npx @redocly/cli@latest respect test-file.yaml --workflow first-flow second-flow

The --workflow option can't be used with --skip.

-s, --skip

[string]

Workflow names from the test file to skip. For example, the following command skips the "first-flow" workflow from the test-file.yaml Arazzo description:

npx @redocly/cli@latest respect test-file.yaml --skip first-flow

Warning

The --skip option can't be used with --workflow.

-v, --verbose

boolean

Runs the command in verbose mode to help with troubleshooting issues. For example, the following command runs all workflows from the test-file.yaml Arazzo description in verbose mode:

npx @redocly/cli@latest respect test-file.yaml --verbose

--har-output

string

Path for the har file for saving logs. For example, the following command runs all workflows from the test-file.yaml Arazzo description and saves the logs to the logs.har file:

npx @redocly/cli@latest respect test-file.yaml --har-output='logs.har'

--json-output

string

Path for the JSON file for saving logs. For example, the following command runs all workflows from the test-file.yaml Arazzo description and saves the logs to the logs.json file:

npx @redocly/cli@latest respect test-file.yaml --json-output='logs.json'

--input

string

Input parameters with values that are mapped to the workflow inputs description. For example, the following command maps the "userEmail" and "userPassword" inputs and values to all workflows in the test.yaml Arazzo description:

npx @redocly/cli@latest respect test.yaml --input userEmail=name@redocly.com --input userPassword=12345

You can also use an environment variable to set the input, as in the following example:

REDOCLY_CLI_RESPECT_INPUT='userEmail=name@redocly.com,userPassword=12345' npm run cli respect test.yaml

You can also include nested values. The following example command maps the nestedKey input and its value to all workflows in the test-file.yaml Arazzo description:

npx @redocly/cli@latest respect test-file.yaml --input '{"key": "value", "nested": {"nestedKey": "nestedValue"}}'

You can also use an environment variable to set the input, as in the following example:

REDOCLY_CLI_RESPECT_INPUT='{"key":"value","nested":{"nestedKey":"nestedValue"}}' npx @redocly/cli@latest respect test-file.yaml

--server

string

Server overrides for the sourceDescriptions object. For example, the following command runs all workflows from the test-file.yaml Arazzo description. Instead of the servers listed in the API description, it uses the specified servers:

npx @redocly/cli@latest respect test-file.yaml --server sourceDescriptionName1=https://test.com --server sourceDescriptionName2=https://test2.com

You can also pass multiple server overrides as an environment variable by separating them with commas, as in the following example:

REDOCLY_CLI_RESPECT_SERVER="sourceDescriptionName1=https://test.com,sourceDescriptionName2=https://test2.com"

--severity

string

Severity of the check. Customize the severity level of validation checks for status codes, schemas, and content types. Setting the severity level is useful when:

  • You're actively developing your API and want to temporarily relax validation.
  • You're testing new API behavior that doesn't match the current schema.
  • You've encountered known issues that shouldn't block test execution.
  • You want to treat certain validation failures as warnings instead of errors.

The following checks can be configured:

  • STATUS_CODE_CHECK: Verifies if the status code returned with API responses matches the statuses described in the provided OpenAPI description.
  • SCHEMA_CHECK: Verifies if the response body schema matches what is defined in the provided OpenAPI description.
  • SUCCESS_CRITERIA_CHECK: Verifies if the success criteria defined in the provided Arazzo description has been met.
  • CONTENT_TYPE_CHECK: Verifies if the Content-Type matches what is defined in the provided OpenAPI description.

The following severity values are available:

  • error: Validation failures cause the workflow to fail.
  • warn: Validation failures appear as warnings but don't fail the workflow.
  • off: Validation checks still run, but their results are ignored.

Setting the severity of a check to off doesn't prevent the check from running. It only stops the check's results from affecting the workflow outcome.

For example, the following command sets status code errors to the warning severity level:

npx @redocly/cli respect test-file.yaml --severity='{"STATUS_CODE_CHECK":"warn"}'

--max-steps

number

Maximum number of steps to run (default: 2000). For example, the following command sets the maximum number of steps to 50:

npx @redocly/cli@latest respect test-file.yaml --max-steps=50

You can also pass the maximum number of steps as an environment variable, as in the following example:

REDOCLY_CLI_RESPECT_MAX_STEPS=50 npx @redocly/cli@latest respect test-file.yaml

--max-fetch-timeout

number

Maximum time to wait for API response per request in milliseconds (default: 40 seconds). For example, the following command sets the maximum fetch timeout to 60 seconds:

npx @redocly/cli@latest respect test-file.yaml --max-fetch-timeout=60000

You can also pass the maximum time to wait as an environment variable, as in the following example:

REDOCLY_CLI_RESPECT_MAX_FETCH_TIMEOUT=60000 npx @redocly/cli@latest respect test-file.yaml

--execution-timeout

number

Maximum time to wait for the Respect execution in milliseconds (default: 1 hour). For example, the following command sets the execution timeout to 30 minutes:

npx @redocly/cli@latest respect test-file.yaml --execution-timeout=1800000

You can also pass the maximum time to wait as an environment variable, as in the following example:

REDOCLY_CLI_RESPECT_EXECUTION_TIMEOUT=1800000 npx @redocly/cli@latest respect test-file.yaml

--no-secrets-masking

boolean

Disables masking of secrets in the output. By default, respect masks sensitive information with ******** in both terminal logs and file outputs. Sensitive information includes values described with format: password, and tokens and authentication headers from x-security.

When this flag is set to true, the raw (unmasked) data shows in all outputs.

For example, the following command disables sensitive output masking:

npx @redocly/cli@latest respect test-file.yaml --no-secrets-masking

Request data from the API description

A step that calls an operation doesn't have to list its parameters or request body. When the step leaves them out, respect takes them from the OpenAPI description:

  • Without parameters, the step sends the operation's required parameters, including the ones defined on the path.
  • Without requestBody, the step sends a body for the first media type of the operation's request body.

Each value comes from the example field, or from the first entry in examples (its value or, in OpenAPI 3.2, its dataValue). Without an example, respect generates a value from the schema.

For example, if the orderId path parameter of getOrderById has the example ord_01h1s5z6vf2mm1mz3hevnn9va7, the following step sends GET /orders/ord_01h1s5z6vf2mm1mz3hevnn9va7:

steps:
  - stepId: get-order
    operationId: getOrderById

A step that lists any parameters takes no parameters from the description. List every required parameter in that step.

Examples

  • Run the tests by running the following command: npx @redocly/cli@latest respect <your-test-file>.
  • Run multiple tests by running the following command: npx @redocly/cli@latest respect <your-test-file-one> <your-test-file-two>.
  • Run multiple tests by running the following command with bash selector : npx @redocly/cli@latest respect $(find ./path-to-tests-folder -type f -name '*.arazzo.yaml').

Example output

Running workflow warp.arazzo.yaml / missionLostInvention

  ✓ POST /anchors - step setAnchorToCurrentTime
    ✓ status code check (Response code 201 matches one of description codes: [201, 409])
    ✓ content-type check
    ✓ schema check

  ✓ POST /timelines - step createTimelineTo1889
    ✓ status code check (Response code 201 matches one of description codes: [201])
    ✓ content-type check
    ✓ schema check

  ✓ POST /travels - step travelTo1889
    ✓ status code check (Response code 200 matches one of description codes: [200, 400])
    ✓ content-type check
    ✓ schema check

  ✓ POST /items - step findAndRegisterBlueprint
    ✓ status code check (Response code 200 matches one of description codes: [200, 409])
    ✓ content-type check
    ✓ schema check

  ✓ POST /paradox-checks - step avoidParadox
    ✓ success criteria check
    ✓ success criteria check
    ✓ status code check (Response code 200 matches one of description codes: [200, 400])
    ✓ content-type check
    ✓ schema check

  ✓ POST /travels - step returnToPresent
    ✓ status code check (Response code 200 matches one of description codes: [200, 400])
    ✓ content-type check
    ✓ schema check


  Summary for warp.arazzo.yaml
  
  Workflows: 1 passed, 1 total
  Steps: 6 passed, 6 total
  Checks: 20 passed, 20 total
  Time: 1060ms


┌──────────────────────────────────────────────────────────┬────────────┬─────────┬─────────┬──────────┬─────────┐
│ Filename                                                 │ Workflows  │ Passed  │ Failed  │ Warnings │ Skipped │
├──────────────────────────────────────────────────────────┼────────────┼─────────┼─────────┼──────────┼─────────┤
│ ✓ warp.arazzo.yaml                                       │ 1          │ 1       │ -       │ -        │ -       │
└──────────────────────────────────────────────────────────┴────────────┴─────────┴─────────┴──────────┴─────────┘

Resources