Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

False negatives in coverage analysis for recursive functions #8679

Open
oranium opened this issue Jul 19, 2024 · 0 comments
Open

False negatives in coverage analysis for recursive functions #8679

oranium opened this issue Jul 19, 2024 · 0 comments
Assignees
Labels
bug Issue is reported as a bug stalled waiting for input by the Erlang/OTP team team:VM Assigned to OTP team VM

Comments

@oranium
Copy link

oranium commented Jul 19, 2024

Describe the bug
Binaries in function heads with no pattern matching seem to not be picked up by cover in recursive functions.
I encountered this bug in an Elixir function when using mix's builtin coverage, but was able to reproduce it with (I hope) equivalent erlang code and using cover directly.

The bug seems to occur when a function header binds to a variable without pattern matching, then recursively calls itself, and
finally terminates in another clause.

When this happens, and coverage information is being collected, the non-binding part of the function is not registered as executed.
I have had this bug happen with a binary specifically.

To Reproduce

I created a small example for this bug and uploaded it to this repo for running the example: cover-recursion

The main code (recursion.erl) contains two functions, covered/1 and notcovered/1, which have identical functionality (yield ok when the binary starts with \n, otherwise remove the first element and recurse if the argument is a binary.

The only difference is one function head: covered(<<Message/binary>>)) leads to the function being registered as executed, while notcovered(Message) when is_binary(message) does not register the message as being executed.
In the example, I added io:fwrite calls to show that notcovered/1 indeed gets executed.

Here is the code to keep everything within the issue.

recursion.erl (The code file)
-module(recursion).
-export([covered/1, notcovered/1]).

covered(<<"\n", _Message/binary>>) ->
  io:fwrite("Running and covered (exit covered/1)\n"),
  ok;

covered(<<Message/binary>>)->
  io:fwrite("Running and covered (recursion covered/1)\n"),
  <<_Start:1/binary, Rest/binary>> = Message,
  covered(Rest).

notcovered(<<"\n", _Message/binary>>) ->
  io:fwrite("Running and covered (exit notcovered/1)\n"),
  ok;

notcovered(Message) when is_binary(Message)->
    io:fwrite("Running but not covered (recursion notcovered/1)\n"),
    <<_Start:1/binary, Rest/binary>> = Message,
    notcovered(Rest).
recursion_SUITE.erl (The test suite)
-module(recursion_SUITE).
-export([test_cover/0, test_nocover/0]).

test_cover() ->
  Res = recursion:covered(<<"f\n">>),
  Res == ok.

test_nocover() ->
  Res = recursion:notcovered(<<"f\n">>),
  Res == ok.
run.erl (Running the code with `cover`)
-module(run).
-export([run/0]).

run() ->
  compile:file("recursion.erl"),
  compile:file("recursion_SUITE.erl"),
  cover:start(),
  cover:compile_module(recursion),
  recursion_SUITE:test_cover(),
  recursion_SUITE:test_nocover(),
  cover:analyse_to_file([html]).

Expected behavior
The executed lines should be registered by cover

Affected versions
I have observed this behavior on OTP-27.0, but not tested it on other versions. I built OTP from source with default options and ran it on WSL (Ubuntu 20.04).

Additional context
I appended the zipped cover output HTML to this issue.
coverage.zip

@oranium oranium added the bug Issue is reported as a bug label Jul 19, 2024
@RaimoNiskanen RaimoNiskanen added the team:VM Assigned to OTP team VM label Jul 23, 2024
@jhogberg jhogberg added the stalled waiting for input by the Erlang/OTP team label Jul 24, 2024
@bjorng bjorng self-assigned this Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue is reported as a bug stalled waiting for input by the Erlang/OTP team team:VM Assigned to OTP team VM
Projects
None yet
Development

No branches or pull requests

4 participants