Met returning 404 from collection API endpoint

[jacobthill]

The Met API we currently use is returning a 404 leading to the following error:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 465, in _execute_task
    result = _execute_callable(context=context, **execute_callable_kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 432, in _execute_callable
    return execute_callable(context=context, **execute_callable_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/baseoperator.py", line 401, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/operators/python.py", line 235, in execute
    return_value = self.execute_callable()
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/operators/python.py", line 252, in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/airflow/dlme_airflow/harvester/source_harvester.py", line 24, in data_source_harvester
    df_and_csv = dataframe_to_file(collection, last_harvest_start_date)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/airflow/dlme_airflow/utils/dataframe.py", line 40, in dataframe_to_file
    source_df = collection.catalog.read()
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/airflow/dlme_airflow/drivers/json.py", line 165, in read
    self._load_metadata()
  File "/home/airflow/.local/lib/python3.12/site-packages/intake/source/base.py", line 84, in _load_metadata
    self._schema = self._get_schema()
                   ^^^^^^^^^^^^^^^^^^
  File "/opt/airflow/dlme_airflow/drivers/json.py", line 145, in _get_schema
    self._open_collection()
  File "/opt/airflow/dlme_airflow/drivers/json.py", line 49, in _open_collection
    ).urls()
      ^^^^^^
  File "/opt/airflow/dlme_airflow/utils/partition_url_builder.py", line 38, in urls
    partition_urls = self._calculate_partitions()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/airflow/dlme_airflow/utils/partition_url_builder.py", line 63, in _calculate_partitions
    record_count = expression.find(self.provider_data)[0].value
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range