[improve][fn] support reading config options from file in Function Python Runner

[tpiperatgod]

Fixes #18533

Master Issue: #18533

Motivation

The Pulsar functions have the ability to provide the number of instances of the called function (see, for example, "Get num instances" ), but the data that this function currently relies on needs to be passed in via command line arguments and cannot be easily changed dynamically. Therefore we need to provide a more dynamic way of accessing data (mainly FunctionDetails) for functions, and this way can also serve to pass the function's other parameters.

This mechanism is aimed at allowing the function to dynamically fetch the required parameters and data without having to restart the function in order to load the latest parameters or data.

A more appropriate approach is to provide the function with a local config file where the data and parameters are stored. The provider of the data and parameters (usually the initiator of the function, e.g. function-worker) writes the contents of the parameters and the data (FunctionDetails) to a fixed local config file, and passes the path of this config file to the function.

exec --config-file /path/to/config-file

The function determines the value of --config-file and determines if the target config file exists, and parses its contents if it does.

The rules for applying parameters or data to the function are as follows.

• Parameters or data passed in on the command line have a lower priority than the corresponding parameters or data in the configuration file
• For mandatory parameters or data, if they are not passed in, an exception is thrown.
• For optional parameters or data, if not passed in, the default value is used

Modifications

1. add two arguments
   • enable_live_update, default is False, which means that python-function will listen for changes to the configuration file and update the configuration of the running function instance when it has listened for an event
   • config_file, default is None, which means you need to get the configuration content from config_file
2. the priority of command line arguments is higher than the priority of configuration file arguments during initialization
3. if args.config_file and args.enable_live_update are enabled, add a new thread and use the watchdog library to listen to args.config_file and update the relevant configuration after listening to the on_modified event

Verifying this change

• Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

• Added integration tests for end-to-end deployment with large payloads (10MB)
• Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository:

[github-actions]

@tpiperatgod Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->