Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not Converting AWS::Glue::Table to aws_glue_catalog_table #320

Open
rknechtel opened this issue Jul 10, 2024 · 1 comment
Open

Not Converting AWS::Glue::Table to aws_glue_catalog_table #320

rknechtel opened this issue Jul 10, 2024 · 1 comment

Comments

@rknechtel
Copy link

cf2tf is not Not Converting a CloudFormation AWS::Glue::Table to aws_glue_catalog_table. It creates it as an aws_route_table.

Example CF Definition:

  VpcFlowLogsAthenafl025c2085d26ae2213daily2024070920240709:
    Type: AWS::Glue::Table
    DependsOn:
      - VpcFlowLogsAthenaDatabase
    Properties:
      CatalogId: !Ref AWS::AccountId
      DatabaseName: !Ref VpcFlowLogsAthenaDatabase
      TableInput:
        Description: This table has the schema for vpc flow logs information.
        Name: fl025c2085d26ae2213daily2024070920240709
        PartitionKeys:
          - Name: year
            Type: string
          - Name: month
            Type: string
          - Name: day
            Type: string
        TableType: EXTERNAL_TABLE
        StorageDescriptor:
          Location:  s3://vpcflowlogs/AWSLogs/123456789012/vpcflowlogs/us-east-1/
          InputFormat: org.apache.hadoop.mapred.TextInputFormat
          OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
          SerdeInfo:
            Parameters:
              skip.header.line.count: "1"
              EXTERNAL: "true"
              field.delim: ' '
              serialization.format: ' '
            SerializationLibrary: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
          Columns:
            - Name: 'version'
              Type: int
            - Name: 'account_id'
              Type: string
            - Name: 'interface_id'
              Type: string
            - Name: 'srcaddr'
              Type: string
            - Name: 'dstaddr'
              Type: string
            - Name: 'srcport'
              Type: int
            - Name: 'dstport'
              Type: int
            - Name: 'protocol'
              Type: bigint
            - Name: 'packets'
              Type: bigint
            - Name: 'bytes'
              Type: bigint
            - Name: 'start'
              Type: bigint
            - Name: 'end'
              Type: bigint
            - Name: 'action'
              Type: string
            - Name: 'log_status'
              Type: string

Output:

resource "aws_route_table" "vpc_flow_logs_athena_table" {
  vpc_id = data.aws_caller_identity.current.account_id
  // CF Property(DatabaseName) = aws_glue_catalog_database.awsdefault_vpc_flow_logs_athena_database.arn
  // CF Property(TableInput) = {
  //   Description = "This table has the schema for vpc flow logs information."
  //   Name = "fl025c2085d26ae2213daily2024070920240709"
  //   PartitionKeys = [
  //     {
  //       Name = "year"
  //       Type = "string"
  //     },
  //     {
  //       Name = "month"
  //       Type = "string"
  //     },
  //     {
  //       Name = "day"
  //       Type = "string"
  //     }
  //   ]
  //   TableType = "EXTERNAL_TABLE"
  //   StorageDescriptor = {
  //     Location = "s3://vpcflowlogs/AWSLogs/123456789012/vpcflowlogs/us-east-1/"
  //     InputFormat = "org.apache.hadoop.mapred.TextInputFormat"
  //     OutputFormat = "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
  //     SerdeInfo = {
  //       Parameters = {
  //         skip.header.line.count = "1"
  //         EXTERNAL = "true"
  //         field.delim = " "
  //         serialization.format = " "
  //       }
  //       SerializationLibrary = "org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"
  //     }
  //     Columns = [
  //       {
  //         Name = "version"
  //         Type = "int"
  //       },
  //       {
  //         Name = "account_id"
  //         Type = "string"
  //       },
  //       {
  //         Name = "interface_id"
  //         Type = "string"
  //       },
  //       {
  //         Name = "srcaddr"
  //         Type = "string"
  //       },
  //       {
  //         Name = "dstaddr"
  //         Type = "string"
  //       },
  //       {
  //         Name = "srcport"
  //         Type = "int"
  //       },
  //       {
  //         Name = "dstport"
  //         Type = "int"
  //       },
  //       {
  //         Name = "protocol"
  //         Type = "bigint"
  //       },
  //       {
  //         Name = "packets"
  //         Type = "bigint"
  //       },
  //       {
  //         Name = "bytes"
  //         Type = "bigint"
  //       },
  //       {
  //         Name = "start"
  //         Type = "bigint"
  //       },
  //       {
  //         Name = "end"
  //         Type = "bigint"
  //       },
  //       {
  //         Name = "action"
  //         Type = "string"
  //       },
  //       {
  //         Name = "log_status"
  //         Type = "string"
  //       }
  //     ]
  //   }
  // }
}

It should create a resource like:
resource "aws_glue_catalog_table" "vpc_flow_logs_athena_table" {

@shadycuz
Copy link
Member

@rknechtel Thanks for reporting this =)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants