Response body data in body post request

[rgraham7503]

I do not see a way to use a previous response body data value in a new request body.
example:
probe id 1 gets an access token
probe id 2 wants to do an introspect on the access token from probe id 1.
Introspect endpoint requires 3 elements in the body
body:
token: tokenvalue
client_id: "xxxx"
client_secret: "xxxx"

I am trying to use the access token from the response body of probe id 1 in the body of probe id2
body:
token: {{response.[0].data.access_token}}
client_id: "xxxx"
client_secret: "xxxx"

This spits out an error :
_err = new cli_2.CLIError(input, options);
^
CLIError: missed comma between flow collection entries (75:31)

72 | timeout: 7000
73 | url: https://xxxx/oidc/endpoint/def ...
74 | body:
75 | token: {{{response.[0].data.access_token}}}_

[dennypradipta]

Could you please provide the full configuration? You may redact sensitive values. And what version of Monika did you use?

If we take a look at the example:

probes:
   - id: '1'
     name: Probing Github
     description: simulate html form submission
     interval: 10
     requests:
       - method: GET
         url: https://reqres.in/api/users
         timeout: 7000
       - method: GET
         url: https://reqres.in/api/users/{{ responses.[0].data.data.[0].id }}
         timeout: 7000
     incidentThreshold: 3
     recoveryThreshold: 3
     alerts:
       - status-not-2xx
       - response-time-greater-than-2000-ms

There are two request inside a probe. The first request will return this:

{"page":1,"per_page":6,"total":12,"total_pages":2,"data":[{"id":1,"email":"[email protected]","first_name":"George","last_name":"Bluth","avatar":"https://reqres.in/img/faces/1-image.jpg"},{"id":2,"email":"[email protected]","first_name":"Janet","last_name":"Weaver","avatar":"https://reqres.in/img/faces/2-image.jpg"},{"id":3,"email":"[email protected]","first_name":"Emma","last_name":"Wong","avatar":"https://reqres.in/img/faces/3-image.jpg"},{"id":4,"email":"[email protected]","first_name":"Eve","last_name":"Holt","avatar":"https://reqres.in/img/faces/4-image.jpg"},{"id":5,"email":"[email protected]","first_name":"Charles","last_name":"Morris","avatar":"https://reqres.in/img/faces/5-image.jpg"},{"id":6,"email":"[email protected]","first_name":"Tracey","last_name":"Ramos","avatar":"https://reqres.in/img/faces/6-image.jpg"}],"support":{"url":"https://reqres.in/#support-heading","text":"To keep ReqRes free, contributions towards server costs are appreciated!"}}

The second request will get the first ID of the data from the response body. So the second request will be reqres.in/api/users/1

{"data":{"id":1,"email":"[email protected]","first_name":"George","last_name":"Bluth","avatar":"https://reqres.in/img/faces/1-image.jpg"},"support":{"url":"https://reqres.in/#support-heading","text":"To keep ReqRes free, contributions towards server costs are appreciated!"}}

[rgraham7503]

Hi @dennypradipta Here is the configuration file with two issues.
body like this that does not work
body:
token: {{ responses.[0].data.access_token }}
client_id: "xxxxxxxx"
client_secret: "xxxxxxxxx"

AND url that does not work.
method: DELETE
timeout: 7000
url: https://xxxxx.com/v2.0/Users/{{ responses.[1].data.id }}
Everything else in this file works.

notifications:
  - id: desktop
    type: desktop
  - id: email notification
    type: smtp
    data:
      recipients: [xxxxxxx]
      hostname: xxxxxx
      port: 587
      username: xxxxxxx
      password: xxxxxx
probes:
  - id: isv_auth
    name: ISV Authentication Monitor
    interval: 30
    requests:
      - method: POST
        timeout: 7000
        url: https://xxxxx.com/v1.0/endpoint/default/token
        body:
          grant_type: "client_credentials"
          client_id: "xxxxxxxx"
          client_secret: "xxxxxxxxx"
        headers:
          Content-Type: application/x-www-form-urlencoded
        alerts:
          - query: response.time > 1200
            message: Login API Response time is {{ response.time }} ms, expecting less than 1200>
          - query: response.status < 200 or response.status > 299
            message: Target is not healthy. It has not been returning status code 2xx.
      - method: POST
        timeout: 7000
        url: https://xxxxx.com/v2.0/Users
        body:
          schemas:
            - urn:ietf:params:scim:schemas:core:2.0:User
            - urn:ietf:params:scim:schemas:extension:ibm:2.0:User
            - urn:ietf:params:scim:schemas:extension:enterprise:2.0:User
            - urn:ietf:params:scim:schemas:extension:ibm:2.0:Notification
          userName: xxxxxxx
          title: Important person
          name:
            familyName: tester
            givenName: tester
            middleName: test
          urn:ietf:params:scim:schemas:extension:ibm:2.0:Notification:
            notifyType: NONE
          displayName: xxxxxx
          preferredLanguage: en-US
          active: true
          emails:
            - value: xxxxxxx
              type: work
          addresses:
            - locality: xxxx
              country: US
              type: work
              streetAddress: 111 my street. mytown, mystate
              postalCode: 'xxxxx'
              primary: true
              region: xx
          phoneNumbers:
            - value: 'xxxxxxxx'
              type: mobile
          urn:ietf:params:scim:schemas:extension:ibm:2.0:User:
            userCategory: regular
            twoFactorAuthentication: false
          urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:
            department: IT Engineering
            employeeNumber: '123234'
          password: xxxxxxxxx
        timeout: '5000'
        headers:
          Content-Type: application/scim+json
          Authorization: Bearer {{ responses.[0].data.access_token }}
        alerts:
          - query: response.status < 201 or response.status > 299
            message: Target is not healthy. It has not been returning status code 2xx.
          - query: response.time > 2800
            message: Login API Response time is {{ response.time }} ms, expecting less than 2000>
            
      - method: GET
        timeout: 7000
        url: https://xxxxx.com/v2.0/Users?filter=userName+eq+%22xxxxxx%22
        headers:
          Content-Type: application/x-www-form-urlencoded
          Authorization: Bearer {{ responses.[0].data.access_token }}
        alerts:
          - query: response.time > 2800
            message: Login API Response time is {{ response.time }} ms, expecting less than 2000>
          - query: response.status < 200 or response.status > 299
            message: Target is not healthy. It has not been returning status code 2xx.
            
      - method: DELETE
        timeout: 7000
        url: https://xxxxx.com/v2.0/Users/{{ responses.[1].data.id }}
        headers:
          notifyType: NONE
          Authorization: Bearer {{ responses.[0].data.access_token }}
        body: {}
        alerts:
          - query: response.time > 2800
            message: Login API Response time is {{ response.time }} ms, expecting less than 2000>
          - query: response.status != 204
            message: Login API Status code is not 204.
      - method: POST
        timeout: 7000
        url: https://xxxxx.com/v1.0/authnmethods/password/xxxxx?returnJwt=true
        body:
          username: "userthatwasjustcreated"
          password: "passwordofuserjustcreated"
        headers:
          Content-Type: application/json
          Authorization: Bearer {{ responses.[0].data.access_token }}
        alerts:
          - query: response.time > 2800
            message: Login API Response time is {{ response.time }} ms, expecting less than 2000>
          - query: response.status != 200
            message: Login API Status code is not 200. Please check the service status!
      - method: POST
        timeout: 7000
        url: https://xxxxx.com/oidc/endpoint/default/introspect
        body:
          token: {{ responses.[0].data.access_token }}
          client_id: "xxxxxxxx"
          client_secret: "xxxxxxxxx"
        headers:
          Content-Type: application/x-www-form-urlencoded
        alerts:
          - query: response.time > 1200
            message: Login API Response time is {{ response.time }} ms, expecting less than 1200>
          - query: response.status < 200 or response.status > 299
            message: Target is not healthy. It has not been returning status code 2xx.
    incidentThreshold: 1
    recoveryThreshold: 1

[rgraham7503]

Git wont let me add the yaml file and when I insert code its fine if its a simple insert, but when I insert full yaml it breaks out of code insert?? not sure how to get you the full yaml in code form

[dennypradipta]

Hmmm, are your API by chance returning the access_token like this?

{ "data": { "access_token": "eyJ......."} }

If yes, then you may need to access it by responses.[0].data.data.access_token

[rgraham7503]

I do not have a problem anywhere else using access token..but this is the json response so you can see...
{ "access_token": "QNWcqrts1Zk93j6ljuZ1szDYnknlfCCFbgcjoCPg", "grant_id": "0ec2459a-2032-4631-a83c-cd8c0a862ceb", "token_type": "Bearer", "expires_in": 7200 }

[rgraham7503]

I only have an issue in the body and I have an issue with using response values in the DELETE url request

[dennypradipta]

After meddling around with the source code, I found that only the headers and URLs can be used with the previous request's response.

In the probing function inside the src/components/probe/probing.ts:

export async function probing(
  requestConfig: Omit<RequestConfig, 'saveBody' | 'alert'>,
  responses: Array<ProbeRequestResponse>
): Promise<ProbeRequestResponse> {
  // Compile URL using handlebars to render URLs that uses previous responses data
  const { method, url, headers, timeout, body, ping } = requestConfig
  const newReq = { method, headers, timeout, ping }
  const renderURL = Handlebars.compile(url)
  const renderedURL = renderURL({ responses }) <== Render the values from the previous requests' response to URL
  let requestBody: any = body

  // Compile headers using handlebars to render URLs that uses previous responses data.
  // In some case such as value is not string, it will be returned as is without being compiled.
  // If the request does not have any headers, then it should skip this process.
  if (headers) {
    for (const header of Object.keys(headers)) {
      const rawHeader = headers[header]
      const renderHeader = Handlebars.compile(rawHeader)
      const renderedHeader = renderHeader({ responses }) <== Render the values from the previous requests' response to Headers

      newReq.headers = {
        ...newReq.headers,
        [header]: renderedHeader,
      }

      // evaluate "Content-Type" header in case-insensitive manner
      if (header.toLocaleLowerCase() === 'content-type') {
        const { content, contentType } = transformContentByType(body, rawHeader)

        // change request body with the transformed request body
        requestBody = content <== This did not render the values from the previous requests' response to URL

        if (rawHeader === 'multipart/form-data') {
          // delete the previous content-type header and add a new header with boundary
          // it needs to be deleted because multipart/form data needs to append the boundary data
          // from
          //    "content-type": "multipart/form-data"
          // to
          //    "content-type": "multipart/form-data; boundary=--------------------------012345678900123456789012"
          delete newReq.headers[header]

          newReq.headers = {
            ...newReq.headers,
            'content-type': contentType,
          }
        }
      }
    }
  }
....

Only the headers and URLs with the previous requests response can be rendered into values, while the body isn't. I think we need to create a ticket to handle this

[rgraham7503]

Oh that would be an awesome enhancement!!!

[dennypradipta]

I've created a ticket #725, it's just a matter of time to solve this problem :)

Thank you for the feedback!

[rgraham7503]

Absolutely! Monika is an awesome product, I'm excited to see the growth and the maturity!

[kevin-hyperjump]

Hi folks, let's goooooo #738 🚀

[sapiderman]

Hi @rgraham7503

We've added this missing functionality. Let us know what you think!