Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go prefixedwriter: fix short write error case #68

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ChenJesse
Copy link

@ChenJesse ChenJesse commented Jun 30, 2023

In certain cases, formatting as JSON actually reduces the number of bytes.
This leads to the bufio.Writer that wraps this returning short write error
because the number of bytes returned is less than expected.

We just want to return the original number of bytes, since we know that we are
doing some string manipulation and we are OK with that.

See: https://pkg.go.dev/io#Writer

Write writes len(p) bytes from p to the underlying data stream. It returns the
number of bytes written from p (0 <= n <= len(p)) and any error encountered
that caused the write to stop early. Write must return a non-nil error if it
returns n < len(p).

how did I test this?

I noticed that sometimes log lines would get lost. with some investigation, i realized that if taskrunner invokes

x := 1
fmt.Println(x)

fmt.Println("should get here")

"should get here" would not be reached because we do the special json formatting which somehow truncates the number of bytes, and we were getting this short write error. This was not apparent because we swallow a lot of Write() errors and such, but I don't want to blow up this PR scope with fixing + testing all of those.

which lead me down to this investigation path.

after this fix, the log lines get printed properly.

In certain cases, formatting as JSON actually reduces the number of bytes.
This leads to the bufio.Writer that wraps this returning `short write` error
because the number of bytes returned is less than expected.

We just want to return the original number of bytes, since we know that we are
doing some string manipulation and we are OK with that.

See: https://pkg.go.dev/io#Writer
```
Write writes len(p) bytes from p to the underlying data stream. It returns the
number of bytes written from p (0 <= n <= len(p)) and any error encountered
that caused the write to stop early. Write must return a non-nil error if it
returns n < len(p).
```
@ChenJesse
Copy link
Author

i think technically we shouldn't be modifying p, but that seems like a much larger design decision, so I would like to patch this for now

@er9781
Copy link
Contributor

er9781 commented Jul 1, 2023

Example before/after? Or at a minimum a "how did I test this"?

@ChenJesse
Copy link
Author

@er9781 Added a "how did I test this" section!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants