Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STAGING -> MASTER #4021

Merged
merged 52 commits into from
Jul 5, 2023
Merged

STAGING -> MASTER #4021

merged 52 commits into from
Jul 5, 2023

Conversation

mat-if
Copy link
Contributor

@mat-if mat-if commented Jul 5, 2023

Summary

Testing Plan

Documentation

Does this change require any updates to the Iron Fish Docs (ex. the RPC API
Reference
)? If yes, link a
related documentation pull request for the website.

[ ] Yes

Breaking Change

Is this a breaking change? If yes, add notes below on why this is breaking and
what additional work is required, if any.

[ ] Yes

hughy and others added 30 commits June 15, 2023 16:10
the poolMiner only updates its target when it receives a 'set_target' message,
and not when it receives new work from the pool.

the pool only sends 'set_target' when the miner first connects, so a pool miner
uses the same target for the duration of its connection.
the poolMiner only updates its target when it receives a 'set_target' message,
and not when it receives new work from the pool.

the pool only sends 'set_target' when the miner first connects, so a pool miner
uses the same target for the duration of its connection.

the miner outputs a log message with its target each time it receives new work,
but this is misleading.

updates the pool miner message to output a slice of the header sent for new
work, which matches the log output of the pool when it sends new work.
* fixes usage of oclif table sort flag

in many of our CLI command that use oclif tables we include the 'sort' flag.
however, if we are displaying streamed responses in a table then the sort flag
doesn't work because each response is a separate table.

- removes 'sort' flag from commands where it can't be used
- adds message if 'sort' is used along with 'follow' and can't be supported in
  peers commands
- removes default setting of 'STATE' for sort in peers command: the table flags
  are global, so this was being applied to every command that includes the sort
  flag

Relates to #3944

* updates to use 'exclusive' feature to prevent follow being used with sort

* restores newline
* Add perf test reports in CI

* Update output path

* Add test:perf command

* Remove flags

* Use jest-junit test reporter

* Rename

* Update path

* Add permission to github action

* Set fail on error to true

* Upload test report

* Increase perf test timeout

* Create new cmd test:perf:report

* Refactor transaction perf tests

* Only run transaciont

* Add test baseline metrics and refactor tests

* Increase timeout"

* Avoid teardown DB after each test case

* List all test cases in report

* Use JUnit Report Action for github action because the previous one does not have detailed report for each test case

* Use Publish Test Results for github action

* Show test result output

* Use default json file

* Add json output formatter"

* Reduce output

* Print only test cases output

* Only print test cases

* Update thredshold
* chore(): Add section for docs to PR template

* Fix error message when chain head changes (#3972)

* Remove the check in perf tests

---------

Co-authored-by: Rohan Jadvani <[email protected]>
Co-authored-by: Daniel Cogan <[email protected]>
* verifies block template after streaming

optimistically sends new block templates to connected clients (miners and
pools) before block verification.

we currently verify a block before sending the template out. however, if the
block verification fails, then the node will crash.

block verification takes time, and miners could be mining on an old template or
an empty template while verification runs. instead, we can send out the
template(s) as quickly as possible and verify afterwards.

- makes block verification optional in newBlock
- does not verify block for empty block template
- verifies 'full' template after sending it out to clients
- re-sends empty template if verification of full template fails

* Update ironfish/src/blockchain/blockchain.ts

avoids buffer comparison when possible

Co-authored-by: mat-if <[email protected]>

---------

Co-authored-by: mat-if <[email protected]>
Found this one util file did not adhere to our convention so I made it
consistent.
This new command will attempt to open your data dir in your gui file
explorer.
* handles errors sending jobs to worker threads

the worker pool tracks the state of a job by assigning it a status and by
assigning it to a worker. that assignment, placing the job in the worker's jobs
map, occurs before sending the job to the worker. the job's status is also set
to 'executing' before sending the job to the worker thread.

if an error occurs in sending the job to the worker, such as when serializing
the job, then we do not reset these statuses: the job is left in an 'executing'
state even though it is not executing, and the worker keeps the job in its map
even though it never received it.

at this point the worker pool will not assign any more jobs to that worker.

- catches errors from 'worker.send' and sets the job status to 'error'
- moves assignment of the job to the worker's data structures to 'job.execute'
  and only assigns the job if it was successfully sent to the worker thread.
- removes 'worker.execute' since it only calls 'job.execute'

* updates test of job execute
* Change Wallet.send() to accept an object

This brings it inline with our other APIs, and makes it more extensible.
This is in preparation for adding another argument to this function
too. It also makes it more backwards compatible for consumers of the
SDK, because adding new parameters won't break existing users.

* Update tests

* fixes test

* fixes lint

---------

Co-authored-by: Hugh Cunningham <[email protected]>
some node runners have seen telemetry jobs failing when a telemtry attempts to
serialize a field that's expected to be an integer.

rounds numeric values for integer fields.

adds additional details to error messages for TypeErrors when serializing
telemetry fields.
* Add influx annotation to test report for upload"

Improve test report data

Console log in non verbose form

clean ci

* Add influx import

* Remove influx token"

* Add necessary annotation in csv for uploading

* Set group key for all test report attributes

* Set token

* Use segment result instead of renderSegment

* Remove trim

* Update annotation
* WIP

* Revert changes to transaction perf test

* Revert changes to package.json

* Revert transaction perf test fixture

* Fix warnings

* Update workflow

* Try again

* Format and clippy

* update test due to change in scalar randomness

The reason this changed is due to how bls12_381 and blstrs generate
random Scalars

* remove opencl dependency

remove opencl dependency to minimize external system dependencies for
now. we can investigate adding gpu support later, but expect most usage
to be on cloud servers without access to gpus

* fix(rust,zkp): Utilize portable flag for blstrs

Blstrs builds using CPU instructions that may not be present on older
hardware. Github Actions hardware pool has some of this older hardware,
leading to broken builds in some cases.

---------

Co-authored-by: Derek Guenther <[email protected]>
Co-authored-by: Derek Guenther <[email protected]>
* adds event/onReorganizeChain RPC

streams the block headers for the old head, the new head, and header for the
point of the fork: the block at which the two forks diverged

adds an event to the blockchain reorganizeChain method to emit these headers on
each reorg

supports monitoring chain reorgs over time

* drops the 'Chain' from the onReorganize event
* PR to see perf metric after blstrs

* Move perf test to separate workflow

* Include the new workflow

* Track only staging
adds a loop to the stats command to listen to a stream of reorganizeChain events
from the node and submit reorg data to telemetry.

submits the hash and sequence of the old head, the new head, and the block at
which the forks diverged.
All the parameters are optional so the body should be optional.
moves account, account tests to new wallet/account directory

defines AccountEncoder type interface
This RPC did not support passing feeRate to this RPC. I restructured it
so it's now supported.
@mat-if mat-if requested a review from a team as a code owner July 5, 2023 17:50
jowparks and others added 9 commits July 5, 2023 14:22
* feat(ci): Add a lint check for PRs against master

This will fail if the PR has a base branch of master and does not
contain the word master in the PR title. This still allows release PRs
and other emergency PRs, while creating a quick and easy check to make
sure PRs aren't accidentally going into master.

* rename job for easier diagnosis
* adds separate RpcAccountImport type

pushes serialization of createdAt as HeadValue to the RPC; serialization of the
block hash as a hex string is only needed for sending RPC responses

makes createdAt type uniform across AccountImport and AccountValue

closes IFL-1259

* fixes lint error

* updates AccountImport to RpcAccountImport in import cli
* implements bech32 account encoder

encodes account data as a buffer by concatenating fields together before
encoding the hex string using bech32.

decoding strings that don't match the encoding returns null

copies buffer encoding pattern from walletdb/accountValue

adds EncodingError to bufio typedefs

* addresses pr feedback

- uses hardcoded encoder version instead of account schema version
- returns null if decoded version does not match encoder version
- uses flag for each optional field instead of a single bitflag
- moves all optional fields to end of encoded string
- uses shorter prefix 'ifaccount' for bech32 strings

* removes unnecessary export

* updates bech32 decode to error instead of return null
* feat: Add monitoring for libuv queue size

* use npm package libuv-monitor

* use isActive flag for libuv-monitor so we can use it on all platforms

* only show 'STOPPED' and not the extra bits if the monitor is not active
We use node version 18, but our node-app is stuck using version 16. This
just formalizes what we already know. We need to support at least
version 16 for now.
@mat-if mat-if merged commit e27a127 into master Jul 5, 2023
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants