Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating Native Dashboards via Terraform Using Layout #208

Open
qgervacio opened this issue Feb 25, 2024 · 3 comments
Open

Creating Native Dashboards via Terraform Using Layout #208

qgervacio opened this issue Feb 25, 2024 · 3 comments

Comments

@qgervacio
Copy link

qgervacio commented Feb 25, 2024

Hi CX team. This is not really a bug but an inquiry. I would like to seek help regarding creations of native dashboards via TF.

The self-service I would like to expose to my users is to create the dashboard first (in any folder), export as JSON and drop to a self-service repo. structure of the repo below

parent-1/sub-11/db.json
parent-2/sub-21/db.json

these folders are created in CX accordingly, however Im stuck in creating the dashboard.

Creating via below seems inflexible(?) as i need to give my users options which folder the dashboard should be created as structured in the self-service repo.

resource "coralogix_dashboard" dashboard_from_json {
  content_json = file("./dashboard.json") # this already have ids and folderId
  
  # perhaps we can allow to override folder id here?
  # folder_id = ...
}

Affected Resource(s)

  • coralogix_dashboard

Terraform Configuration Files

terraform {
  required_version = "~> 1.7"
  required_providers {
    ...
    coralogix = {
      source  = "coralogix/coralogix"
      version = "1.11.12"
    }
  }
}
...

Debug Output

Panic Output

Expected Behavior

Dashboard is created in a folder

Actual Behavior

Inappropriate value for attribute "layout": attribute "sections": element 0: attribute "rows": element 0: attribute "height" is required.

Steps to Reproduce

i have created a native dashboard using the UI and exported the JSON file.

{
  "id": "1mDArBIW4knyUoHUYNCDo",
  "name": "test",
  "layout": {
    "sections": [
      {
        "id": {
          "value": "856d1938-7e1b-fd5a-36a3-83e8bef6de32"
        },
        "rows": [
          {
            "id": {
              "value": "fafd7374-801b-6dbb-57c6-8da617b3133a"
            },
            "appearance": {
              "height": 19
            },
            "widgets": [
              {
                "id": {
                  "value": "c8442aa5-e3a3-2cae-ed90-c7b7145afc93"
                },
                "title": "New line chart",
                "definition": {
                  "lineChart": {
                    "legend": {
                      "isVisible": true,
                      "columns": [],
                      "groupByQuery": true
                    },
                    "tooltip": {
                      "showLabels": false,
                      "type": "TOOLTIP_TYPE_ALL"
                    },
                    "queryDefinitions": [
                      {
                        "id": "f9cd41a1-2fff-590a-acc2-7e2b46324169",
                        "query": {
                          "logs": {
                            "groupBy": [],
                            "aggregations": [
                              {
                                "count": {}
                              }
                            ],
                            "filters": [],
                            "groupBys": []
                          }
                        },
                        "seriesCountLimit": "20",
                        "unit": "UNIT_UNSPECIFIED",
                        "scaleType": "SCALE_TYPE_LINEAR",
                        "name": "Query 1",
                        "isVisible": true,
                        "colorScheme": "cold",
                        "resolution": {
                          "bucketsPresented": 96
                        },
                        "dataModeType": "DATA_MODE_TYPE_HIGH_UNSPECIFIED"
                      }
                    ]
                  }
                },
                "appearance": {
                  "width": 0
                }
              }
            ]
          }
        ]
      }
    ]
  },
  "variables": [],
  "filters": [
    {
      "source": {
        "logs": {
          "operator": {
            "equals": {
              "selection": {
                "list": {
                  "values": []
                }
              }
            }
          },
          "observationField": {
            "keypath": [
              "applicationname"
            ],
            "scope": "DATASET_SCOPE_LABEL"
          }
        }
      },
      "enabled": true,
      "collapsed": false
    },
    {
      "source": {
        "logs": {
          "operator": {
            "equals": {
              "selection": {
                "list": {
                  "values": []
                }
              }
            }
          },
          "observationField": {
            "keypath": [
              "subsystemname"
            ],
            "scope": "DATASET_SCOPE_LABEL"
          }
        }
      },
      "enabled": true,
      "collapsed": false
    }
  ],
  "relativeTimeFrame": "900s",
  "folderId": {
    "value": "52bc7c39-3f38-4871-9559-a8aecb674285"
  },
  "annotations": [],
  "off": {}
}

as part of my TF, i run a removal of certain fields.

// ❯ tf fmt && tf console
// > local.file_paths
// toset([
//   "plat/test_02-25-2024.json",
// ])
resource "null_resource" "clean_json" {
  for_each = { for i, v in local.file_paths : i => v }

  triggers = {
    file_paths = each.value
  }

  provisioner "local-exec" {
    command = <<-EOT
      cat ${local.cx_src_dir}/${each.value} | jq 'del(.. | .off?, .folderId?, .id?)' > ${local.cx_src_dir}/${each.value}.json-edited
      rm -rf ${local.cx_src_dir}/${each.value}
      mv ${local.cx_src_dir}/${each.value}.json-edited ${local.cx_src_dir}/${each.value}
    EOT
  }
}

and below is the sample result of this null_resource.clean_json

{
  "name": "test",
  "layout": {
    "sections": [
      {
        "rows": [
          {
            "appearance": {
              "height": 19
            },
            "widgets": [
              {
                "title": "New line chart",
                "definition": {
                  "lineChart": {
                    "legend": {
                      "isVisible": true,
                      "columns": [],
                      "groupByQuery": true
                    },
                    "tooltip": {
                      "showLabels": false,
                      "type": "TOOLTIP_TYPE_ALL"
                    },
                    "queryDefinitions": [
                      {
                        "query": {
                          "logs": {
                            "groupBy": [],
                            "aggregations": [
                              {
                                "count": {}
                              }
                            ],
                            "filters": [],
                            "groupBys": []
                          }
                        },
                        "seriesCountLimit": "20",
                        "unit": "UNIT_UNSPECIFIED",
                        "scaleType": "SCALE_TYPE_LINEAR",
                        "name": "Query 1",
                        "isVisible": true,
                        "colorScheme": "cold",
                        "resolution": {
                          "bucketsPresented": 96
                        },
                        "dataModeType": "DATA_MODE_TYPE_HIGH_UNSPECIFIED"
                      }
                    ]
                  }
                },
                "appearance": {
                  "width": 0
                }
              }
            ]
          }
        ]
      }
    ]
  },
  "variables": [],
  "filters": [
    {
      "source": {
        "logs": {
          "operator": {
            "equals": {
              "selection": {
                "list": {
                  "values": []
                }
              }
            }
          },
          "observationField": {
            "keypath": [
              "applicationname"
            ],
            "scope": "DATASET_SCOPE_LABEL"
          }
        }
      },
      "enabled": true,
      "collapsed": false
    },
    {
      "source": {
        "logs": {
          "operator": {
            "equals": {
              "selection": {
                "list": {
                  "values": []
                }
              }
            }
          },
          "observationField": {
            "keypath": [
              "subsystemname"
            ],
            "scope": "DATASET_SCOPE_LABEL"
          }
        }
      },
      "enabled": true,
      "collapsed": false
    }
  ],
  "relativeTimeFrame": "900s",
  "annotations": []
}

then in dashboard creation

// ❯ tf fmt && tf console
// > local.json_files
// {
//   "plat" = {
//     "content" = {
//       "annotations" = []
//       "filters" = [
//         {
//           "collapsed" = false
//           "enabled" = true
//           "source" = {
//             "logs" = {
//               "observationField" = {
//                 "keypath" = [
//                   "applicationname",
//                 ]
//                 "scope" = "DATASET_SCOPE_LABEL"
//               }
//               "operator" = {
//                 "equals" = {
//                   "selection" = {
//                     "list" = {
//                       "values" = []
//                     }
//                   }
//                 }
//               }
//             }
//           }
//         },
//         {
//           "collapsed" = false
//           "enabled" = true
//           "source" = {
//             "logs" = {
//               "observationField" = {
//                 "keypath" = [
//                   "subsystemname",
//                 ]
//                 "scope" = "DATASET_SCOPE_LABEL"
//               }
//               "operator" = {
//                 "equals" = {
//                   "selection" = {
//                     "list" = {
//                       "values" = []
//                     }
//                   }
//                 }
//               }
//             }
//           }
//         },
//       ]
//       "layout" = {
//         "sections" = [
//           {
//             "rows" = [
//               {
//                 "appearance" = {
//                   "height" = 19
//                 }
//                 "widgets" = [
//                   {
//                     "appearance" = {
//                       "width" = 0
//                     }
//                     "definition" = {
//                       "lineChart" = {
//                         "legend" = {
//                           "columns" = []
//                           "groupByQuery" = true
//                           "isVisible" = true
//                         }
//                         "queryDefinitions" = [
//                           {
//                             "colorScheme" = "cold"
//                             "dataModeType" = "DATA_MODE_TYPE_HIGH_UNSPECIFIED"
//                             "isVisible" = true
//                             "name" = "Query 1"
//                             "query" = {
//                               "logs" = {
//                                 "aggregations" = [
//                                   {
//                                     "count" = {}
//                                   },
//                                 ]
//                                 "filters" = []
//                                 "groupBy" = []
//                                 "groupBys" = []
//                               }
//                             }
//                             "resolution" = {
//                               "bucketsPresented" = 96
//                             }
//                             "scaleType" = "SCALE_TYPE_LINEAR"
//                             "seriesCountLimit" = "20"
//                             "unit" = "UNIT_UNSPECIFIED"
//                           },
//                         ]
//                         "tooltip" = {
//                           "showLabels" = false
//                           "type" = "TOOLTIP_TYPE_ALL"
//                         }
//                       }
//                     }
//                     "title" = "New line chart"
//                   },
//                 ]
//               },
//             ]
//           },
//         ]
//       }
//       "name" = "test"
//       "relativeTimeFrame" = "900s"
//       "variables" = []
//     }
//     "name" = "test"
//   }
// }
resource "coralogix_dashboard" "dashboard" {
  for_each = { for k, v in local.json_files : k => v }
  name     = each.value.name
  folder = {
    id = coralogix_dashboards_folder.parents[each.key].id
  }

  layout = {
    sections = each.value.content.layout.sections
  }
}
# returns 'Inappropriate value for attribute "layout": attribute "sections": 
# element 0: attribute "rows": element 0: attribute "height" is required.'

Important Factoids

References

@OrNovo
Copy link
Contributor

OrNovo commented Feb 26, 2024

Hi @qgervacio If I understand correctly you are trying to create a dashboard resource from a json imported from the UI. The problem is that the structure of the json and the resource's schema are not completely overlap.

@qgervacio
Copy link
Author

i have observed the ff behaviour when using below:

create a dashboard in UI, export to test.json

resource "coralogix_dashboard" "dashboard" {
  content_json = file("./test.json")
}
  1. cannot use the file as is. need to remove off or you'll get json content is not matching layout schema. got an err while unmarshalling - proto: (line 125:3): unknown field "off"
  2. if the manually created dashboard is still in UI, you'll get id already exist.
  3. delete the dashboard in UI first. test.json is created successfully
  4. delete the dashboard in UI first, delete all id in test.json you get dashboard id not found

seems to me that with content_json would make that kind of self-service i want difficult to achieve.

@rudolf-repcin
Copy link

rudolf-repcin commented Apr 18, 2024

I had the same issue, this exported JSON configuration is not meant to be used with Terraform but rather for importing the dashboards in UI.

What I found to be working more or less is creating the dashboard manually in UI, then using a "data" source in Terraform to load its config to tfstate and then taking that TF config and pasting to my code. However there are IDs included that are read-only - you need to remove them.

Would be a nice feature though - to export TF config from the UI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants