Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(agent): use Observer API to hook into Zend Engine for PHPs 8.0+ #542

Merged
merged 68 commits into from
Mar 1, 2024

Commits on Nov 1, 2022

  1. Configuration menu
    Copy the full SHA
    fa73a84 View commit details
    Browse the repository at this point in the history
  2. feat(agent): Registered function begin/end and error handlers with Ob…

    …server API (#501)
    
    * feat(agent): Registered function begin/end and error handlers with Observer API
    1) Registered function begin/end handlers with Observer API
    2) Created the function begin/end handler stubs.  Full functionality is schedule for another ticket.
    3) Registered currently existing error handler with Observer API
    4) created php_observer.c/h module to contain the observer api logic.
    
    Testing:
    1) Verified new function begin/end handlers were registered correctly and received zend_execute_data from PHP engine.
    2) Current test cases verified that registering our current error handler directly caused no change in functionality.
    
    Additional
    * feat(agent): Don't call handlers for internal functions.
    Currently internal functions are not handled by OAPI, but they will be in 8.2.  These functions are tailored to USER functions (similar to nr_php_execute) and we don't want internal functions filtered to these handlers.  This will default to INTERNAL functions being handled by the current implementation.  To change in the future, it's possible we'd need to implement handlers specific for internal functions (similar to nr_php_execute_internal).
    zsistla authored and lavarou committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    a3c4cb5 View commit details
    Browse the repository at this point in the history
  3. feat(agent): Add/Update functions to utilize the OAPI zend_execute_da…

    …ta. (#505)
    
    * feat(agent): Add/Update functions to utilize the OAPI zend_execute_data.
    
    1) Added 4 new functions for use with OAPI instrumentation:
    extern const char* nr_php_zend_execute_data_function_name(
        const zend_execute_data* execute_data);
    
    extern const char* nr_php_zend_execute_data_filename(
        const zend_execute_data* execute_data);
    
    extern const char* nr_php_zend_execute_data_scope_name(
        const zend_execute_data* execute_data);
    
    extern uint32_t nr_php_zend_execute_data_lineno(
        const zend_execute_data* execute_data);
    2) Added test cases to test new functionality.
    3) Updated the following to reflect new OAPI paradigm and add additional checks for robustness:
    * #define NR_PHP_USER_FN_THIS()
    * nr_php_execute_scope
    3) Added unit test cases to test new functions
    4) Also verified functionality via integration tests.
    
    * refactor(agent): If called correctly, new functions should work with PHP 7+.
    zsistla authored and lavarou committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    73d2816 View commit details
    Browse the repository at this point in the history
  4. feat(agent): Propagate OAPI return values and update return value fun…

    …ctions. (#516)
    
    * feat(agent): Propagate OAPI return values and update return value functions.
    1) updated macros to pass the OAPI given return value throughout the userland system.
    2) changed nr_php_get_return_value to return oapi given pointer
    3) Added the OVERWRITE_ZEND_EXECUTE_DATA to allow us to toggle off during feature addition, but toggle off to maintain CI as long as possible.  It also gives the flexibility to revert to the instrumentation prior to OAPI.
    4) macro to toggle between just using the OAPI return value when OAPI is enabled or calling nr_php_get_return_value_ptr when it is not OAPI.
    zsistla authored and lavarou committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    5365878 View commit details
    Browse the repository at this point in the history
  5. feat(agent): update nr_get_zend_execute_data to only use OAPI provide…

    …d zed. (#517)
    
    * feat(agent): Propagate OAPI return values and update return value functions.
    1) updated macros to pass the OAPI given return value throughout the userland system.
    2) changed nr_php_get_return_value to return oapi given pointer
    3) Added the OVERWRITE_ZEND_EXECUTE_DATA to allow us to toggle off during feature addition, but toggle off to maintain CI as long as possible.  It also gives the flexibility to revert to the instrumentation prior to OAPI.
    4) macro to toggle between just using the OAPI return value when OAPI is enabled or calling nr_php_get_return_value_ptr when it is not OAPI.
    zsistla authored and lavarou committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    2871e2f View commit details
    Browse the repository at this point in the history
  6. feat(agent): Add code level metrics functionality. (#506)

    * feat(agent): Add code level metrics functionality.
    1) Add INI value to disable/enable code level metrics.
    2) Create new function `nr_php_txn_add_code_level_metrics` that uses nr_php_zend_execute_data_* family of functions to extract CLM and add it as an `agent attribute`.
    3) Added new NR_TXN_ATTRIBUTES for `code.namespace`, `code.lineno`, `code.filepath`, and `code.function`.
    4) Added call to php_execute_enabled to call the new CLM function
    4) Added new integration tests to exercise the new functionality.
    
    Note:  CLM functionality is only compatible with PHP 7+.
    
    * feat(agent): Code level metrics functionality.
    1) updated `nr_php_execute_metadata_t` to hold additional information and moved the definition of the struct to php_execute.h.
    2) `nr_php_execute_enabled` uses the metadata at the beginning so now we can simply release at the end.
    3) In the case of a super short segment that would have been ignored, we do NOT add CLM (otherwise it wouldn't be ignored).  We wait until after we decide to ignore or not to add the CLM data.
    
    * feat(agent): Updated test scripts.
    
    * fix(agent): check clm string lengths will not be truncated
    
    * fix(agent): check for empty/null CLM attributes
    
    * style(agent): clang-format php_execute
    
    * fix(agent): fix logic handling CLM string length
    
    * refactor(agent): abstract out CLM while-loop code for readability
    zsistla authored and lavarou committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    ecd977c View commit details
    Browse the repository at this point in the history
  7. feat(agent): user instrumentation via PHP's OAPI (#551)

    Features:
    * implement user functions instrumentation via PHP's Observer API
    * remove function hooks that overwrite `zend_execute_ex` for user
    functions
    * add `special_instrumentation_before` callback to `nruserfn_t` for user
    functions that need instrumentation to happen before the function
    executes.
    * add `nr_php_wrap_user_function_before_after` to install Observer API's
    before and after function callbacks (to be used in lib_*.c and/or fw_*.c)
    * add `nr_zend_call_oapi_special_before` to call Observer APIs
    before function callback (to be used in `observer_fcall_begin` handler).
    * store txn_start_time in a segments to so that it is available in
    `observer_fcall_end` handler to verify and end the segment
    * add reportedclass to nruserfn_t to account for functions existing in
    one class table while reporting they belong to another class (details
    commented in code).
    
    Refactorings:
    * add `nr_php_wraprec_matches` helper function
    * add `wraprec` to segments to only make the call to get_wraprec_by_name
    once (in the `observer_fcall_begin` handler and not again in the
    `observer_fcall_end` handler)
    
    Tests:
    * update unit tests to test new features and changed functionality
    zsistla authored and lavarou committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    6d632da View commit details
    Browse the repository at this point in the history
  8. feat(agent): implement php_execute_show functionality for OAPI (#555)

    * feat(agent): implement php_execute_show functionality for OAPI
    
    * Update agent/php_execute.c
    
    Co-authored-by: Michal Nowacki <[email protected]>
    
    * style(agent): PR feedback, remove comment
    
    * Update agent/php_execute.c
    
    Co-authored-by: Michal Nowacki <[email protected]>
    bduranleau-nr and lavarou committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    e74dfc0 View commit details
    Browse the repository at this point in the history
  9. fix(agent): detect framework in oapi's fcall_begin (#554)

    Framework detection (nr_execute_handle_framework) not only adds wraprecs
    but also sets NRPRG(current_framework). It is important to set it before
    function is called because its value affects other instrumentation, e.g.
    when datastore segment for SQL operation is ended, a table name modify
    function (nr_php_modify_table_name_fn) to shorten table name is selected
    based on NRPRG(current_framework).
    
    This fixes frameworks/magento/test_temp_tables.php and
    frameworks/wordpress/test_site_specific_tables.php integration tests.
    lavarou committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    8db0cab View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    4c732d9 View commit details
    Browse the repository at this point in the history
  11. fix(agent): fix check for max strlen when generating clm attributes (#…

    …563)
    
    * fix(agent): fix check for max strlen when generating clm attributes
    
    * style(tests): follow php function naming convention (camelCase)
    bduranleau-nr authored and lavarou committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    363983d View commit details
    Browse the repository at this point in the history
  12. fix(agent): instrument drupal_http_request (#552)

    Fix `drupal_http_request` instrumentation for PHP 8.0+ by using Observer
    APIs `before` callback to add New Relic headers and `after` callback
    to finalize external segment with metrics. Observer API instrumentation
    requires the external request segment to be created in `before` callback
    but available in `after` callback. Therefore it is made a NRPRG global.
    lavarou committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    5896966 View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2023

  1. feat(agent): OAPI exception handling (#580)

    1. Renamed nr_php_execute_metadata_add_code_level_metrics to
    nr_php_observer_metadata_init as this will be the way we populate the
    metadata. We don't need to do duplicate effort with
    `nr_php_execute_metadata_init` anymore.
    2. Since nr_php_execute_metadata_init is now for populating metadata,
    moved all CLM checks out nr_php_observer_metadata_init and into
    nr_php_txn_add_code_level_metrics where it is more appropriate.
    3. Modified stacked segments to have a pointer to the metadata so if
    they are closed off by an exception, they will properly propogate the
    information. Added initializations/deninitializations.
    4. For php_execute_enabled, only call nr_php_observer_metadata_init if
    we need to.
    5. Added nr_php_observer_exception_segment_end
     To handled closing off observer segments if an exception occurs.
    6. Now use zend_throw_exception_hook (more info:
    https://www.phpinternalsbook.com/php7/extensions_design/hooks.html
    Note: This ONLY notifies when an exception is thrown. It gives no
    indication if that exception was subsequently caught or not.
    8. OAPI leaves dangling segments in the case of exceptions. These need
    to be cleaned up for functions that rely on the current segment
    (includes begin/end functions, stacked segment unwinding, and API calls)
    9. New inline function re`nr_php_api_ensure_current_segment` to account
    for dangling segments when calling our API.
    10. TXN globals to keep track of the exception
    11. Change in php_txn turn off recording after unwind the segments to
    give timer to attach exception to dangling segment(s).
    12. Modify stacked segments init/denit to handle additional segment
    metadata variable.
    13. Functions to clear/set TXN uncaught_exception variables.
    14. Update metadata struct to retain more context of the segment.
    15. Removed legacy exception code that wasn't getting called anymore.
    16. Added tests.
    17. Added a `clean`callback to the wraprec functionality and associated
    unit tests.
    
    With OAPI exceptions, the registered function handler (nr_php_observer_fcall_end) doesn't get called when an uncaught exception occurs, and therefore doesn't decrement the stack_depth counter.
    
    All OAPI unhandled exception cleanup filters through: nr_php_observer_segment_end so we decrement php_cur_stack_depth there when we cleanup orphaned segments.
    Additionally, since nr_php_observer_segment_end is an exit path, also call nr_php_show_oapi_metadata
    New function nr_php_show_oapi_metadata called via the segment exception handling exit path (to correspond to nr_php_show_exec_return) to show the all the available function details when the special_flags.show_execute_* is toggled on. This will help when debugging.
    Added additional test cases to ensure proper php_cur_stack_depth counting
    zsistla authored Jan 20, 2023
    Configuration menu
    Copy the full SHA
    4893277 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c7d4dfc View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2023

  1. fix(agent): move globals that utilized the call stack into a global s…

    …tack when using OAPI (#582)
    
    Symfony1 instrumentation utilizes the call stack, but should never be
    run with PHP8+.
    ZNeumann authored Jan 26, 2023
    Configuration menu
    Copy the full SHA
    8764ea6 View commit details
    Browse the repository at this point in the history
  2. feat(agent): Remove php_vm.c overwriting opcodes when using OAPI (#559)

    Occasionally, we want to instrument functions called with
    call_user_func_array, when we otherwise wouldn't instrument that
    function. To do this, we instrument the internal function
    call_user_func_array and check for contexts in which we want to begin
    instrumentation. However, call_user_func_array can be inlined by the
    zend compiler. Previously, we were detecting inlined calls by
    overwriting the DO_FCALL opcode. With OAPI, we no longer want to touch
    the opcodes and are instead checking for the inlined calls during
    observer_do_fcall.
    ZNeumann authored Jan 26, 2023
    Configuration menu
    Copy the full SHA
    0051e67 View commit details
    Browse the repository at this point in the history
  3. fix(agent): Ensure txn exists before we try to reference it. (#600)

    Consistent with other functions in php_execute, check if txn exists
    before trying to use it.
    
    The issue occurred during the first request, and because it is the
    first, the appname is unknown and we don't create a txn. However, an
    exception happened in that first request, our exception_hook handler got
    triggered, and if that is triggered, we always try to close off dangling
    segments. In this case, because we didn't check for txn==null, it
    segfaulted as we tried to access txn elements.
    Pr fixes that in two ways, 1) check for txn==null before closing off
    segments. 2) don't record uncaught exception info generated from our
    exception_hook handler if a txn is null.
    ```
    2023-01-26 21:03:16.816 +0000 (29813 29813) verbosedebug: RINIT processing started
    2023-01-26 21:03:16.816 +0000 (29813 29813) debug: added app='PHP Test Apps Laravel ads v2' license='07...4a'
    2023-01-26 21:03:16.816 +0000 (29813 29813) verbosedebug: querying app='PHP Test Apps Laravel ads v2' from parent=4
    2023-01-26 21:03:16.816 +0000 (29813 29813) verbosedebug: sending appinfo message, len=6492
    2023-01-26 21:03:16.817 +0000 (29813 29813) debug: APPINFO reply unknown app='PHP Test Apps Laravel ads v2'
    2023-01-26 21:03:16.817 +0000 (29813 29813) debug: unable to begin transaction: app 'PHP Test Apps Laravel ads v2' is unknown
    ```
    note the last line: debug: 
    unable to begin transaction: app 'PHP Test Apps Laravel ads v2' is
    unknown
    zsistla authored Jan 26, 2023
    Configuration menu
    Copy the full SHA
    4ea50b6 View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2023

  1. fix(wordpress): fix unbound memory allocation when wordpress tags off (

    …#601)
    
    move NR_GET_RETURN_VALUE_PTR before NR_PHP_WRAPPER_CALL
    Cleanup conditional compilation to be more readable
    ZNeumann authored Jan 27, 2023
    Configuration menu
    Copy the full SHA
    947f42a View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2023

  1. fix(agent): Explicitly call some special function instrumentation in …

    …oapi func_begin for txn naming. (#615)
    
    For OAPI, wrapping default makes all wrapped special functions execute
    during func_end. This caused issues since to customize txn naming per
    framework, PHP agent uses the order in which functions
    are processed, NR_NOT_OK_TO_OVERWRITE/NR_OK_TO_OVERWRITE, and whether it
    is
    called either before or after NR_PHP_WRAPPER_CALL (for pre PHP 8+) or
    whether
    it is called in func_begin or func_end (for PHP 8+ / OAPI). By
    defaulting to calling all callbacks "after" during func_end, it changed
    the order in which nr_txn_set_path was being set. To fix this, we need
    to verify which callback functions set the txn before
    NR_PHP_WRAPPER_CALL and which set it after, and have the wrapped
    functions act similarly to call before/after.
    
    
    This PR 
    1) reviewed all frameworks (exceptions listed below) and use of
    nr_txn_set_path on a case by case basis.
    2) Explicitly commented to detail the naming schemes in use with each
    use of nr_txn_set_path in all frameworks
    3) used the before/after/clean wrapping function as needed when txn
    naming needed to occur before function execution to match what is done
    in legacy instrumentation.
    4) for functions not needing to be called before, comments were still
    added noting that it was being called by the default (after
    callback/func_end)
    
    Calling the wrapper special function by default before a function
    executes was a solution initially considered; however, after
    investigation/testing, it was deemed inappropriate for multiple reasons:
    1)too invasive 
    2)some wrapped functions do not have all the values they need before a
    function is executed, for example
    functions that need the return value (I.e. all those special functions
    that continue to do things after `NR_PHP_WRAPPER_CALL`
    3) Additionally, each framework has individualized, specialized ways of
    setting the txn name via `nr_txn_set_path`.
    
    As such, calling a wrapper function in func_end as default is still the
    most appropriate choice.
    
    These frameworks are either unsupported, do not support PHP 8+, or both
    and no change to the txn naming scheme:
    fw_kohana.c, fw_joomla.c, fw_magneto.c(magento 1.x), fw_silex.c,
    fw_symfony.c, fw_symfony2.c, fw_zend.c fw_zend2.c
    
    Unit tests that run on all versions of PHP ensure the txn naming
    behavior is identical.
    zsistla authored Feb 16, 2023
    Configuration menu
    Copy the full SHA
    7648b1a View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2023

  1. Configuration menu
    Copy the full SHA
    295ab5d View commit details
    Browse the repository at this point in the history

Commits on Mar 9, 2023

  1. Configuration menu
    Copy the full SHA
    c974a2e View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2023

  1. test(agent): Fix tests to CLM on (#637)

    Modify newly added oapi tests to conform with CLM on by default.
    mfulb authored Mar 10, 2023
    Configuration menu
    Copy the full SHA
    f3c38a3 View commit details
    Browse the repository at this point in the history
  2. chore(install): Disable ZTS installs (#636)

    Removes support for ZTS installs in `newrelic-install.sh` as disables
    building ZTS artifacts.
    
    ---------
    
    Co-authored-by: Amber Sistla <[email protected]>
    mfulb and zsistla authored Mar 10, 2023
    Configuration menu
    Copy the full SHA
    4209f55 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2023

  1. Revert "chore(install): Disable ZTS installs" (#662)

    Reverts #636. This change is obsoleted by
    #661. Revert is needed to avoid conflicts when dev is merged into oapi.
    lavarou authored Apr 28, 2023
    Configuration menu
    Copy the full SHA
    cd99428 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2023

  1. Configuration menu
    Copy the full SHA
    712396c View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2023

  1. tests: 'enhance' logging libraries mocks (#699)

    Empty files don't get executed by PHP 8.2 when the agent uses Observer
    API to hook into Zend engine, therefore mocks needs to do something -
    enhance them with a 'noop' statement - `echo "";`
    lavarou authored Jul 26, 2023
    Configuration menu
    Copy the full SHA
    21a383b View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. tests: fix predis/test_pipeline expectation (#704)

    restore expected values updated in
    329acaa which got overwritten with
    c974a2e.
    lavarou authored Aug 1, 2023
    Configuration menu
    Copy the full SHA
    447daab View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. suppress Invalid read error in test_redis (#707)

    A call to `zend_map_ptr_reset` in `zend_deactivate` triggers `Invalid
    read of size 8` valgrind error in test_redis during
    `tlib_php_request_eval_expr` and `nr_php_zval_free` in the last php
    request lifecycle when Observer API is used to hook into Zend engine.
    This error is triggered on all PHPs with this patch:
    php/php-src@ff62d11
    lavarou authored Aug 8, 2023
    Configuration menu
    Copy the full SHA
    fdb79fd View commit details
    Browse the repository at this point in the history
  2. fix(agent): set oapi callbacks the right way (#700)

    When wrapping a user function do not add special instrumentation if ANY
    special instrumentation (before/after/cleanup) has already been set.
    This approach matches pre-oapi instrumentation.
    ZNeumann authored Aug 8, 2023
    Configuration menu
    Copy the full SHA
    33fed1d View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2023

  1. feat(agent): instrument Drupal 9.4 hooks with oapi (#701)

    In addition to OAPI Drupal 9.4 support, this PR does some refactoring:
    
    1. The hook stacks have been renamed from `module_invoke_all` to
    `invoke_all` to be more aligned with the name of Drupal functions that
    can now push to those stacks.
    
    2. The generic wrapper function (originally added when Drupal 9.4
       instrumentation was introduced) has been updated to support OAPI.
    
    3. `nr_php_wrap_callable_before_after_clean` has been introduced as a
       parallel for `nr_php_wrap_user_function_before_after_clean`. To
    facilitate this, many of the internals have been extracted to a common
    function.
    
    4. Speaking of, lots of the maintenance of the Drupal's hook stacks has
       been extracted to common functions to reduce code reuse
    
    ---------
    
    Co-authored-by: Michal Nowacki <[email protected]>
    ZNeumann and lavarou authored Aug 9, 2023
    Configuration menu
    Copy the full SHA
    6846ba4 View commit details
    Browse the repository at this point in the history
  2. fix: add guards to oapi cufa opcode check (#708)

    When instrumenting wordpress/drupal hooks, we want to instrument the
    hook function calls, which are dispatched by php's
    call_user_function_array. This php function (cufa for short) is
    flattened/inlined, so to determine if we are in a function called by a
    cufa call, we need to check the opcodes of the caller to the current
    function. If the caller is a php internal function, checking these
    opcodes was not safe and lead to invalid reads. As such, before reading
    the opcodes, we first check if the caller is a user function. We only
    want to instrument cufa calls from user functions anyway.
    
    ---------
    
    Co-authored-by: Michal Nowacki <[email protected]>
    ZNeumann and lavarou authored Aug 9, 2023
    Configuration menu
    Copy the full SHA
    dca53bd View commit details
    Browse the repository at this point in the history
  3. fix(agent): wrap zend_try in a func to avoid clobbering (#703)

    Calls to `call_user_function` (in PHP 8.0 and 8.1) and
    `zend_call_method_if_exists` (in PHP 8.2+) need to be wrapped by the
    `zend_try`/`zend_catch`/`zend_end_try` block, which use
    [`setjmp`](https://linux.die.net/man/3/setjmp) and
    [`longjmp`](https://linux.die.net/man/3/longjmp), because according to
    [`call_user_func()`](https://www.php.net/manual/en/function.call-user-func.php):
    > Callbacks registered with functions such as `call_user_func()` and
    [`call_user_func_array()`](https://www.php.net/manual/en/function.call-user-func-array.php)
    will not be called if there is an uncaught exception thrown in a
    previous callback.
    
    So if we call something that causes an exception, it will block us from
    future calls that use `call_user_func` or `call_user_func_array`.
    Valgrind showed the agent and/or the Zend engine wasn’t properly
    cleaning up after such cases and newer compilers had issues with this
    when compiling with any optimization and generated the following error:
    ```
    error: variable ‘retval’ might be clobbered by ‘longjmp’ or ‘vfork’) [-Werror=clobbered]
    ```
    PHP developers solve this problem by using an automatic variable to
    store user function call result - see
    [here](https://github.com/php/php-src/blob/master/main/streams/userspace.c#L335-L340)
    for an example in PHP source code how zend_call_method_if_exists is
    called.
    
    This solution wraps `zend_try`/`zend_catch`/`zend_end_try` constructs in
    a function to localize the `retval` and avoid variable clobbering.
    ZNeumann authored Aug 9, 2023
    Configuration menu
    Copy the full SHA
    1e7c861 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2023

  1. tests: 'enhance' laravel framework mock (#716)

    Bypass PHP 8.2 optimization of not executing files without executable
    statements by forcing the execution of the file with mock of Laravel
    used by the test, so when wrapper for
    `Illuminate\Routing\RouteCollection::getRouteForMethods` is installed,
    the `Illuminate\Routing\RouteCollection` class and `getRouteForMethods`
    can be found.
    lavarou authored Aug 21, 2023
    Configuration menu
    Copy the full SHA
    7d0a768 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2023

  1. Configuration menu
    Copy the full SHA
    8e683b1 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. tests: fix metrics expectation in drupal tests (#726)

    External metrics should not be generated when call to
    `drupal_http_request` fails due to uncaught error. This works correctly
    in PHPs 8+, when Observer API is used to hook into Zend Engine but
    doesn't work correctly for previous method of hooking into Zend Engine
    hence two flavors of tests.
    lavarou authored Sep 6, 2023
    Configuration menu
    Copy the full SHA
    f582376 View commit details
    Browse the repository at this point in the history
  2. refactor(agent): improve phpunit instrumentation (#725)

    * improve phpunit detection
    Use file that is loaded also on PHP 8.2 with include/require
    optimizations that don't execute files without executable
    statements.
    
    * improve skipped tests handling
    PHPUnit passes skipped tests to `addError` as well as `endTest`
    regardless ofthe version. The difference is that in PHPUnit 5.x it
    is not possible to obtain test outcome for skipped test - 
    `$test.getStatus()` returns 'null'. Because of that, `endTest` 
    instrumentation bails out and does not generate test event and
    therefore `addError` needs to be instrumented. It is possible
    to obtain test outcome for skipped tests in PHPUnit 9.x. However,
    the `$time` for skipped test is not received in `endTest` as a valid
    double and therefore it needs to be 'fixed'. Moreover 
    testcase.getStatusMessage returns 'null' for tests skipped due to
    dependency failure and therefore it is not possible to expect it in
    the test event.
    lavarou authored Sep 6, 2023
    Configuration menu
    Copy the full SHA
    7b142fc View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2023

  1. Configuration menu
    Copy the full SHA
    fae2aed View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. Configuration menu
    Copy the full SHA
    f9fa14a View commit details
    Browse the repository at this point in the history
  2. feat: optional instrumentated function metric (#730)

    There still exists a default wrapper, `nr_php_wrap_user_function`, which
    (for OAPI) creates an 'after' wraprec that is non-transient and creates
    an `InstrumentedFunction` metric. Before this PR, there was the option
    to create 'before', 'after', and 'clean' wrappers and define the
    transience.
    
    This PR extends the options available when creating a wraprec to no
    longer create the InstrumentedFunction metric. It wraps this option with
    the other bool option (transience) into an options struct.
    
    Additionally, if the top-most default, `nr_php_wrap_user_function`, is
    not used, every single option must be defined ('before', 'after',
    'clean', transience, and instrumented metric). This takes the concept
    that if a single one of these fields is non-default, it is much more
    likely that other fields will also be non-default. The code writer will
    be forced to explicitly consider each option available.
    
    This does not add the option to create the InstrumentedFunction metric
    when calling `nr_wrap_callable`.
    ZNeumann authored Sep 21, 2023
    Configuration menu
    Copy the full SHA
    54a23d4 View commit details
    Browse the repository at this point in the history
  3. feat: redo wordpress hooks instrumentation for OAPI (#731)

    Bypass the need to check for cufa calls during do_action/filter by
    instrumenting add_action/filter and wrapping the desired calls at hook
    time, rather than call time. Substantially improves performance and
    stability.
    ZNeumann authored Sep 21, 2023
    Configuration menu
    Copy the full SHA
    b6c47aa View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2023

  1. Configuration menu
    Copy the full SHA
    4e791b7 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2023

  1. fix(agent): fix wrapping transient user functions (#735)

    Prevent adding transient wraprecs for already wrapped transient
    functions by checking if wraprec already exists in the hashmap.
    Additionally prevent adding transient wraprecs for if the transient
    function is already wrapped as non-transient function by checking if
    wraprec already exists in the linked list.
    
    For non-transient wrappers (standard instrumentation), the wrapper
    is stored in both the hashmap and linked-list. For transient wrappers,
    the wrapper is only stored in the hashmap. HOWEVER! non-transient
    wrappers MAY only be in the linked-list. This normally happens if it
    is wrapping a function that hasn't been loaded by PHP yet. Once the
    function is loaded, there are many hooks to ensure the wrapper is also
    added to the hashmap.
    
    The implications of the above are: For non-transient wrappers, we only
    need to check the linked-list to ensure that we are not duplicating a
    wrapper. If a wrapper is only in the hashmap, it is transient and will
    be overwritten by the non-transient wrapper.
    
    For transient wrappers, however, we must check both the linked
    list and the hashmap. This is because it is possible to attempt to
    create a transient wrapper around an already non-transiently wrapped
    function. This will return the non-transient wrapper and will attempt
    to set the transient callbacks if that wrapper has those callbacks free.
    This means that it is possible for callbacks intended to be transient
    are attached onto a wrapper that isn't.
    ZNeumann authored Sep 29, 2023
    Configuration menu
    Copy the full SHA
    3eb2abe View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7b349f5 View commit details
    Browse the repository at this point in the history
  3. add new metric to oapi integration tests

    Add log decorating metric to integration tests added in oapi.
    lavarou committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    2b9e3b4 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2023

  1. tests(agent): Add tests that specifically exercise JIT. (#715)

    Tests to exercise JIT.
    
    Please see original forked PR here:
    #583
    
    ---------
    
    Co-authored-by: Michal Nowacki <[email protected]>
    zsistla and lavarou authored Oct 22, 2023
    Configuration menu
    Copy the full SHA
    8bcd5ca View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2023

  1. tests: fix user exception and error handler implementation (#748)

    * fix user exception handler implementation
    User exception handler, set with set_exception_handler is a callable that needs
    to accept one parameter, which will be the Throwable object that was thrown. If
    that callable is not defined according to these rules, PHP's behavior, and
    therefore agent's behavior, is undefined. E.g. in PHPs 8.0, 8.1 and 8.2, when
    opcache is disabled Zend Engine gives the agent a zval which is an object of
    Throwable type. The same thing happens in PHPs 8.0 and 8.1 when opcache is
    enabled. However, when opcache is enabled in PHP 8.2, Zend Engine no longer
    gives the agent a zval which is an object of Throwable type.
    
    * fix user error handler implementation
    User error handler, set with `set_error_handler` is a callable that needs to
    accept at least two parameters: int errno and string errstr set to error level
    and error message respectively. If that callable is not defined according to
    these rules, PHP's behavior, and therefore agent's behavior, is undefined.
    
    * split integration test
    An agent hooking into Zend Engine using zend_execute (pre-oapi) hook behaves
    differently than the agent hooking into Zend Engine using observer API (oapi).
    Namely pre-oapi agent does not generate error events when an exception is
    handled with user exception handler wheras oapi agent generates error events in
    such case. Therefore the test testing agent's behavior when an exception is
    handled by user exception handler must be split depending on what method is
    used to hook into Zend Engine: zend_execute hook is used for PHPs < 8.0 and
    observer API is used for PHPs >= 8.0.
    lavarou authored Oct 23, 2023
    Configuration menu
    Copy the full SHA
    a0783d4 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2023

  1. fix(agent): Correctly handle laravel queues (#746)

    When run via artisan queue:listen or artisan queue:work, Laravel queue
    jobs are generated as a separate background transaction for each job
    that is processed. These transactions are named with the class name of
    the job (or IlluminateQueueClosure if a closure was queued) and the
    connection type which is stored in the application's config/queue.php
    configuration file that includes connection configurations for each of
    the queue drivers that are included with the framework(sync--for dev env
    to execute immediately, database, [Amazon
    SQS](https://aws.amazon.com/sqs/), [Redis](https://redis.io/), and
    [Beanstalkd](https://beanstalkd.github.io/) drivers, and null --discards
    queued jobs).
    
    For example, current implementation shows a job like:
    `OtherTransaction\/Custom\/App\\Http\\Controllers\\JobOne (database)`
    `OtherTransaction` shows it is a background job
    `App\\Http\\Controllers\\JobOne` is the class name of the job
    `database` is the connection type
    
    
    This PR:
    * Removed WorkCommand::handle wrapper 
    * removed worker::process wrapper
    * removed all non-supported Laravel 4.0 instrumentation.
    * updated link
    * removed references to "fire"
    * removed unneeded job parsing code since the laravel Job class already
    provides the info
    * Added handling to detect when a txn is ended within a wrapped function
    (that could result in segfaults when transactions were restarted (either
    directly through newrelic_start_transaction() or indirectly through
    newrelic_set_appname(), or with laravel queue instrumentation).
    
    1)With laravel updates, there are better functions to instrument queue
    jobs that also result in fewer extraneous segments.
    For OAPI/PHP 8+ and moving forward, we can use:
    * Illuminate\\Queue\\Worker::raiseBeforeJobEvent(string $connectionName,
    Job $job):void
    *   Illuminate\\Queue\\SyncQueue::raiseBeforeJobEvent(Job $job):void
    * Illuminate\\Queue\\Worker::raiseAfterJobEvent(string $connectionName,
    Job $job):void
    *   Illuminate\\Queue\\SyncQueue::raiseAfterJobEvent(Job $job):void
    
    2)Using updated laravel functionality, we can additionally simplify the
    txn name decoding quite a bit and add additional information.
    Current naming is: "job_name (connection_type)"
    
    New naming is: "job_name (connection_type:job_queue)"
    
    3) git diff got confused. 
    For the purpose of reviews, despite git saying anything else, in
    fw_laravel_queue.c all the code above line 159 and the
    #if ZEND_MODULE_API_NO >= ZEND_8_0_X_API_NO \
        && !defined OVERWRITE_ZEND_EXECUTE_DATA
    Block is the same code/functionality as currently exists.
    
    Multiverse PR that tests both sync and async (database) queue workers is
    forthcoming.
    zsistla authored Nov 13, 2023
    Configuration menu
    Copy the full SHA
    58662eb View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2023

  1. Configuration menu
    Copy the full SHA
    069310b View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2023

  1. Configuration menu
    Copy the full SHA
    55d781c View commit details
    Browse the repository at this point in the history
  2. fixup! 24c1c65

    After #766 `nr_execute_handle_framework` needs filename length.
    lavarou committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    b4e3e5f View commit details
    Browse the repository at this point in the history
  3. fixup! 0150c09

    After #778 wordpress cleaned tags need not to be freed - they're memoized in
    a hashmap that persists through the whole request and is destroyed in rshutdown.
    lavarou committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    bdb1174 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2024

  1. tests: remove broken exception handling unit tests (#817)

    PHP embed SAPI, used by unit tests, demonstrates memory issues when PHP
    code executed by way of `nr_php_call` or `tlib_php_request_eval` throws
    a PHP Exception. This functionality is best suited to be exercised via
    integration test.
    bduranleau-nr authored Jan 25, 2024
    Configuration menu
    Copy the full SHA
    99bd0ef View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2024

  1. Configuration menu
    Copy the full SHA
    93e5a54 View commit details
    Browse the repository at this point in the history
  2. fixup! 43590ae

    After #798 there's no need to fix wrapping of transient user functions in
    nr_php_add_custom_tracer_named because it is no longer used for those.
    lavarou committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    bfc5ac2 View commit details
    Browse the repository at this point in the history
  3. fixup! 43590ae

    After #798 there's no need to pass any wraprec options when wraprecs are
    created with nr_php_add_custom_tracer_named because it is no longer used
    to create different types of wraprecs. It always creates non-transient
    wraprecs and therefore always needs to generate instrumented function metrics.
    lavarou committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    5035298 View commit details
    Browse the repository at this point in the history
  4. fixup! 0fa3083

    After #768, filter hook's callback function instrumentation needs to create
    wordpress metrics when hook callback function throws an exception. Therefore
    its 'after' wrappers (after and clean) need to be set to do it on hook's
    callback function either normal return or when it returns via an exception.
    lavarou committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    a20529b View commit details
    Browse the repository at this point in the history
  5. fixup! ac78511

    After #799, wordpress hooks stack handlers need to be able to generate hooks
    metrics too when hooks are to be monitored but not hooks' callback functions,
    i.e. when newrelic.framework.wordpress.hooks.options=threshold.
    lavarou committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    e99f7cb View commit details
    Browse the repository at this point in the history
  6. fixup! 43590ae

    After #798, the tests expect Drupal\page_cache\StackMiddleware\PageCache::get to
    be instrumented so that transient instrumentation will not get installed. But
    empty files don't get executed by PHP 8.2+ when the agent uses Observer API to
    hook into Zend engine, therefore mocks needs to do something in order for the
    non-transient instrumentation to be installed (classes and class' methods need
    to be available at the time non-transient instrumentation is installed). Enhance
    Drupal\page_cache\StackMiddleware\PageCache with a 'noop' statement - echo "";`
    lavarou committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    53d10e0 View commit details
    Browse the repository at this point in the history
  7. fix(agent): revert zend_try/catch php call logic (#811)

    zend_try/zend_catch is to handle Zend Exceptions not PHP exceptions -
    see
    [here](https://github.com/newrelic/newrelic-php-agent/blob/320ea571a11bc469d7d8179dfe51577b54df11df/agent/php_user_instrument.c#L17-L50)
    for more details. They were added in this
    [commit](66eccf7)
    to handle misbehaving PHP embed SAPI that threw Zend Exception when PHP
    code that was called via nr_php_call threw PHP Exception. PHP CLI or CGI
    SAPIs don’t throw Zend Exception when PHP code that was called threw PHP
    Exception and therefore zend_try/zend_catch is effectively a dead code.
    bduranleau-nr authored Jan 31, 2024
    Configuration menu
    Copy the full SHA
    4da1211 View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2024

  1. feat(oapi): rework exception handling (#767)

    Removing the need to manual handling of dangling segments, because Zend
    calls all of the hooks we need.
    
    - adds a boolean argument to nr_php_error_record_exception to control
    whether we add the error to the current segment. This is needed because
    the OAPI context of when the above is called is no longer during a
    segment with an uncaught exception and was incorrectly adding the error
    to the root segment.
    - removes all language of "dangling segments". These no longer exist.
    Zend calls all of the necessary `fcall_end`'s, even when an exception is
    thrown. When this happens,` func_return_value` is a C `NULL` pointer
    which is distinct from a `NULL` (but valid) zval when there is no return
    value from a function. We use this `NULL` value to determine the
    presence of an uncaught exception.
    - no longer overwrites the exception hook; no longer stores a copy of
    exceptions locally
    - replace storing metadata in the segment, which was used to pair
    segments from func_begin with func_end, with logic that always creates
    segment in func_begin
    
    Mostly undoes the work of
    #580
    
    ---------
    
    Co-authored-by: Michal Nowacki <[email protected]>
    Co-authored-by: bduranleau-nr <[email protected]>
    Co-authored-by: Amber Sistla <[email protected]>
    4 people authored Feb 2, 2024
    Configuration menu
    Copy the full SHA
    5640be3 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2024

  1. refactor: condense exception handling instrumentation (#825)

    Move exception handler instrumentation from fcall_begin to fcall_end.
    This requires a change of how we are obtaining the fields of the
    exception, because calling `nr_php_call` during the handling of an
    exception does not play nicely with PHP.
    
    ---------
    
    Co-authored-by: Michal Nowacki <[email protected]>
    ZNeumann and lavarou authored Feb 5, 2024
    Configuration menu
    Copy the full SHA
    49a6c1a View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. feat(oapi): use segment pool instead of stacked segments (#776)

    Because OAPI doesn't create a nice callstack that replicates the PHP
    callstack, stacked segments make little/no sense in its context.
    Instead, we want to utilize the segment pool available in txn's.
    
    force_current_segment was how stacked segments were able to maintain
    their context in a txn while not being on the segment stack. For OAPI,
    there is no reason to use this field, as the current segment will just
    be the top of the txn's segment stack.
    
    ---------
    
    Co-authored-by: bduranleau-nr <[email protected]>
    Co-authored-by: Michal Nowacki <[email protected]>
    3 people authored Feb 6, 2024
    Configuration menu
    Copy the full SHA
    26648fe View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5d7ee9b View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. feat: faster segment de-parenting via index tracking (#801)

    Segment order does not matter, as the collector reconstructs the tree
    with timing and parenting info. As such, we can remove segments with
    O(1) speed instead of O(N).
    
    This is an alternative approach than
    #780 and only 1 of
    the 2 should be merged.
    
    ---------
    
    Co-authored-by: Michal Nowacki <[email protected]>
    ZNeumann and lavarou authored Feb 8, 2024
    Configuration menu
    Copy the full SHA
    958d859 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2024

  1. feat(tests): Enable opcache by default for the integration runner. (#752

    )
    
    * Enable the opcache.so extension by default in the integration_runner
    * Ensure the opcache INI settings in the integration_runner environment
    are enabled (these can be overwritten in the test case INI section if
    needing to test without opcache)
    * Added few tests to ensure compatibility with opcache disabled and to
    demonstrate how to overwrite the value.
    
    ---------
    
    Co-authored-by: ZNeumann <[email protected]>
    Co-authored-by: Zach Neumann <[email protected]>
    Co-authored-by: Michal Nowacki <[email protected]>
    4 people authored Feb 22, 2024
    Configuration menu
    Copy the full SHA
    55d3904 View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2024

  1. Configuration menu
    Copy the full SHA
    b38f940 View commit details
    Browse the repository at this point in the history
  2. fixup! e11b992

    fix Drupal package detection
    
    Use file that is loaded also on PHP 8.2 with include/require optimizations that
    don't execute files without executable statements.
    lavarou committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    180bf59 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. fix(agent): fix Lumen detection (#838)

    Remove 'bootstrap/app.php' from the list of signature files used to detect 
    Laravel - Laravel is detected with 'illuminate/foundation/application.php'
    for all supported Laravel versions: 6, 7, 8, 9 and 10.
    Using 'bootstrap/app.php' causes Laravel to be detected before Lumen
    can be detected because 'bootstrap/app.php' is the second file loaded
    when Lumen based app is handling the request and this results in Lumen
    app being detected as Laravel.
    lavarou authored Feb 27, 2024
    Configuration menu
    Copy the full SHA
    4086b6b View commit details
    Browse the repository at this point in the history