Releases: hashgraph/hedera-mirror-node
v0.13.1
v0.13.0
This release is a smaller release mainly focused on bug fixes with some minor enhancements. We added a new property hedera.mirror.importer.downloader.endpointOverride for testing. We also added hedera.mirror.importer.downloader.gcpProjectId to support specifying requester pays credentials with a personal account. Finally, we improved our Marketplace support getting us one closer to making it available.
Enhancements
- Bump versions for v0.13.0-rc1 #815
- Added missing gcpProjectId value to application.yml needed for startup #809
- Update home README Getting Started Instructions #799
- Add Container Support for large db perf tests #797
- Add GCP Marketplace verification #795
- Address review comments in PR 757 #794
- GCP Requester Pays - add project id config #793
- Home README needs Assumed Knowledge & PrerequisiteTool Install Requirements #790
- Add Service Account Configuration to GCP Marketplace Schema #784
- Add Service Account Configuration to GCP Marketplace Schema #776
- Marketplace: Verification Process #737
- Parser performance tests against large db #591
Bug Fixes
- Fix balance file processing non-deterministically #798
- Ensure 404 repsonse for unmatched routes #788
- REST API Not Returning 404 on expected scenarios #751
Documentation
Contributors
We'd like to thank all the contributors who worked on this release!
v0.12.0
This feature release contains a few nice additions while fixing a few critical bugs. We made good progress on adding our application to Google Cloud Platform Marketplace. This should be wrapping up soon and enable a "one click to deploy" of the mirror node to Google's Cloud. Additionally, some extra fields were added to our APIs. We added runningHashVersion to the REST and GRPC APIs. Finally, we added transactionHash to the transaction REST API.
We improved the importer ingestion rate from 3400 to 5600 transactions per second in our performance test environment. There's still room for improvement and we plan on making additional performance optimizations in an upcoming release.
Breaking Changes
We added an option to keep signature files after verification. By default, we no longer store signatures on the filesystem. If you'd like to restore the old behavior and keep the signatures, you can set hedera.mirror.importer.downloader.record.keepSignatures=true and hedera.mirror.importer.downloader.balance.keepSignatures=true.
We changed the bypass hash mismatch behavior in this release. Bypassing hash mismatch could be used in combination with other parameters to fast forward mirror node to newer data or to overcome stream resets. Previously you had to specify this via a database value in t_application_status. Since this data is not application state but considered more a user supplied value, we added a new property hedera.mirror.importer.verfiyHashAfter=2020-06-05T17:16:00.384877454Z for this purpose.
Enhancements
- Bump to v0.12.0 #814
- Cherry pick 794 and bump v0.12.0-rc1 #807
- Bump versions for v0.12.0-beta1 #782
- Enable reWriteBatchedInserts in driver #781
- Fix PostgreSQL network config in marketplace #775
- Add option to archive verified signature files #772
- Removed subscribe flag as acceptance test and updated cucumber version #768
- Configure Postgres Charts for Marketplace deployment #766
- Add runningHashVersion to REST and GRPC APIs #765
- Add consensusTimestamp attribute to PubSub messages #764
- Set default topicRunningHashV2AddedTimestamp based on network #763
- Add consensus_start and consensus_end to t_record_files #762
- Change hash mismatch bypassing from state to config based #757
- Add transactionHash to /transactions REST api #755
- Stop polling bucket when finished downloading demo data #753
- Remove
@SubscribeOnlyFlag Test from@AcceptanceSuite of HCS Tests #752 - Include 3rd party licenses and source in container images #750
- Include 3rd party licenses and source in images #746
- Default topicRunningHashV2AddedTimestamp #745
- Add Application resource for Google Cloud Marketplace #744
- Added deployer folder structure with schema and docker file #740
- Marketplace: Helm Deployer image #736
- Marketplace: Application custom resource #735
- Add consensus_ns of first and last transactions in t_record_files #724
- Add transactionHash to Transaction API #720
- Expose new receipt.topicRunningHashVersion in REST & GRPC API's #707
- HederaETL: Implement deduplication cron job #666
- Archive verified signature files #510
- Unable to change bypass hash mismatch property #249
- Elasticsearch metrics integration #81
Bug Fixes
- Fix docker skip deployer circle ci #783
- Fix chart in GKE #769
- Fix failed crypto transfers with non-fee transfers enabled crashing importer #767
- Update Spring Boot to 2.2.7 to fix NPE in GRPC #748
- NPE in Reactor Netty #739
- Crash importing records into non_fee_transfers #647
Documentation
- Update parser design doc #743
Contributors
We'd like to thank all the contributors who worked on this release!
v0.11.0
This release was mainly focused on refactoring code and properties as a necessary step for future enhancements. We also continued making improvements to our Kubernetes support. To that end, we added Prometheus REST metrics, Helm tests and Mirror Node can now run in GKE.
We added a new parameter to all of the topic related REST APIs to return a topic message in plaintext instead of binary. Messages submitted to HAPI are submitted as binary and stored in the Mirror Node that way as well. If you know the messages are actually strings encoded in UTF-8, then you can set encoding=utf-8 and the REST API will make a best effort conversion to string. By default or if you pass a query parameter of encoding=base64, it will return the message as base64 encoded binary.
Breaking Changes
Please note when upgrading that we made major breaking changes to the naming of our configuration properties. We've renamed all hedera.mirror.api properties to hedera.mirror.rest. We also renamed the properties apiUsername to restUsername and apiPassword to restPassword to reflect that as well. Any properties that were used by the importer module were renamed to be nested under hedera.mirror.importer. We apologize for any inconvenience.
We've removed the hedera.mirror.addressBookPath property in favor of a hedera.mirror.importer.initialAddressBook property. The former was overloaded to be both the initial bootstrap address book and the live address book being updated by file transactions for 0.0.102. The live address book is now hardcoded to ${hedera.mirror.importer.dataPath}/addressbook.bin and cannot be changed.
The REST API to retrieve a topic message by its consensus timestamp now supports both a plural (/topics/messages/:consensusTimestamp) and singular (/topic/message/:consensusTimestamp) URI path. The singular format is deprecated and will be going away in the near future, so please update to the plural format soon.
We removed the singular form of a few alpha topic REST APIs. The /topic/:id/message API was removed in favor of the plural form /topics/:id/messages. Similarly, the /topic/:id/message/:sequencenumber API was removed in favor of its plural form /topics/:id/messages/:sequencenumber. Please update accordingly.
Enhancements
- Prepare for v0.11.0 #760
- Automate versioning and prepare for v0.11.0-rc1 #742
- Auto Clean up old master images in GCR #734
- Lint and template helm charts #731
- Add PubSub integration tests #729
- Homogenize integration test setup #728
- Added running_hash_version migration #726
- Record Parser file reorg #725
- Update protobuf for Hedera Node 0.5.0 #722
- Update protobuf for Hedera Node 0.5.0 #721
- Add initialAddressBook property #719
- Add Helm tests #718
- Fixes to run in GKE #716
- Separate configuration properties by project #715
- Add missing transactionType to PubSubMessge #714
- Change 'left outer join' to 'join' with t_cryptotransferlists #706
- PubSub exporter integration tests and docs #705
- Topic message format support for UTF8 message encoding #704
- Support UTF encoding of Topic Message Results #703
- Configure bootstrap address book path #697
- Consume REST Swagger Prometheus Formatted Metrics #691
- Auto Clean up old master images in GCR #689
- Store new receipt.topicRunningHashVersion #688
- Kubernetes: Manual Deployment to GKE #669
- Kubernetes: Helm chart CI #654
- Kubernetes: Helm chart tests #653
- Nest module properties under a module specific path #416
- Remove the 'left outer join' from the queries in the transactions queries #136
Bug Fixes
- Add awaitjs and swagger missing rest api bundle dependencies #733
- Utilize plural rest api endpoints without breaking support #710
- Topic REST API Endpoints utilize both singular and plural paths #700
Documentation
- Add pubsub docs. Minor properties refactor #741
Contributors
We'd like to thank all the contributors who worked on this release!
v0.10.1
Small bug fix release to address a REST API packaging issue.
Bug Fixes
- Fix REST bundled dependencies and prepare for v0.10.1 #732
Contributors
We'd like to thank all the contributors who worked on this release!
v0.10.0
In preparation for Hedera Node release 0.5.0, we're releasing v0.10.0 to support the latest version of HAPI. The changes include renaming Claims to LiveHash and new response codes. One important HAPI change is the addition of a topicRunningHashVersion to the transaction record. This change was necessary as the way the topic running hash is calculated is changing with the release of 0.5.0. As a result, the Hedera Mirror Node added this new field to its database and a migration is ran to populate it with either the new or old version depending upon the release date of 0.5.0.
Unfortunately, this necessitated adding a required field hedera.mirror.topicRunningHashV2AddedTimestamp to control this behavior and will fail on startup if this is not populated. This is just a temporary measure. Once Hedera Node 0.5.0 is released to testnet and mainnet we will update this so it's automatically populated with the known date.
Other changes include adding Google PubSub support to publish JSON representing the Transaction and TransactionRecord protobuf to a message queue for external consumption. We've also added REST API metrics and added Traefik as an API gateway for our helm chart.
Breaking changes
We've had to remove our event stream support. This area of the code was never enabled and was untested and was incurring technical debt without providing any benefit. If it becomes necessary in the future, we can re-add it within our newly refactored framework.
The new /api/v1/topics/:id alpha REST API that was added in 0.9 has been changed to /api/v1/topics/:id/messages. This change was made to align the API with the other topic message APIs as it refers to the messages entity and not the topic entity.
Enhancements
- Bump versions for v0.10.0 #730
- Port running hash version to V0.10 #727
- Prepare for v0.10.0-rc1 #708
- Add Traefik API gateway #693
- REST API swagger metrics #690
- Pubsub exporter for Mirror Node #686
- Push docker images on master and cleanup old images #683
- Non-fee transfers : move txn type check to implementation #681
- Add updateEntity() and updateTransaction() to TransactionHandler #680
- Store PostgreSQL passwords in a Kubernetes Secret #678
- Remove jdbc connection params not applicable on Postgres connection #671
- Remove Events #670
- HederaETL: PubSub exporter #665
- REST API Metrics #661
- Kubernetes: API Gateway #651
Bug Fixes
- Fix address book updates that span transactions #701
- Topic Message REST API Missing Path #699
- Fix topic messages REST route #698
- Fix Spring Cloud Kubernetes running locally #674
- Implement async rest api routes and centralized error handling #663
- REST API crashes when database error #631
Contributors
We'd like to thank all the contributors who worked on this release!
v0.9.1
Small bug fix release to address not being able to handle address book updates that span multiple transactions.
Bug Fixes
- Address book fix #696
- Rework address book file update #58
- Address book parsing fails when file update spans transactions #695
Contributors
We'd like to thank all the contributors who worked on this release!
v0.9.0
This release contains another new REST API for our consensus service. You can now retrieve all topic messages in a particular topic, with additional filtering by sequence number and consensus timestamp. Here's an example:
GET /api/v1/topic/7?sequencenumber=gt:2×tamp=lte:1234567890.000000006&limit=2
{
"messages": [
{
"consensus_timestamp": "1234567890.000000003",
"topic_id": "0.0.7",
"message": "bWVzc2FnZQ==",
"running_hash": "cnVubmluZ19oYXNo",
"sequence_number": 3
},
{
"consensus_timestamp": "1234567890.000000004",
"topic_id": "0.0.7",
"message": "bWVzc2FnZQ==",
"running_hash": "cnVubmluZ19oYXNo",
"sequence_number": 4
}
],
"links": {
"next": "/api/v1/topic/7?sequencenumber=gt:2×tamp=lte:1234567890.000000006×tamp=gt:1234567890.000000004&limit=2"
}
}The other big feature of this release is Kubernetes support. We've created a Helm chart that can be used to deploy a highly available Mirror Node with a single command. This feature is still under heavy development as we work towards converting our current deployments to this new approach.
Enhancements
- Prepare for v0.9.0 #685
- Bump version for v0.9.0-rc1 #662
- Add pod priority class #650
- Add Pod Priority Class #649
- Add getEntityId() to Transaction Handler #645
- Implement Topic Messages collection REST API endpoint #644
- Add TransactionBody to RecordItem #639
- Kubernetes #637
- Add rest api code coverage #629
- Updated HCS Acceptance tests for easy running by devops #626
- REST API code coverage #623
- Topic messages REST API #618
- Google Blockchain ETL Design #560
- Investigate requester pays buckets #357
- Kubernetes Deployment #346
Bug Fixes
- Fix NPE when parsing autoRenewAccount #682
- 0.9 cherry picks and v0.9.0-rc2 #676
- Fix NPE due to proxyAccountID #664
- Fix REST API packaging issues #648
- Extend timeout in subscribeverifysequence to retrieve messages #643
- Investigate ConsensusControllerTest.subscribeVerifySequence failures #640
- Fixing REST spec tests causing bigint overflow #636
- Store surefire logs in artifacts for debugging of failures #635
- Fix REST Monitor security vulnerabilities #633
- Fix REST security vulnerability #632
- Fix REST tests in CircleCI #630
- CircleCI tests sporadically fail #621
Contributors
We'd like to thank all the contributors who worked on this release!
v0.8.1
Small bug fix release to fix a packaging issue.
Bug Fixes
Contributors
We'd like to thank all the contributors who worked on this release!
v0.8.0
Mirror node v0.8.0 is here! We're made great strides in making the mirror node easier to run and manage. In particular, we added support for requester pays buckets. This will allow anyone to run a mirror node as long as they are willing to pay for the cost to retrieve the data. Currently only Hedera and a few partners have access to the bucket, so enabling this will open up that data to our community. We are still working on a migration of the buckets to this model, so stay tuned.
We also added two new experimental REST APIs to retrieve HCS data. Firstly, we added /api/v1/topic/message/${consensusTimestamp} to retrieve a topic message by its consensus timestamp. Secondly, we added /api/v1/topic/${topic}/message/${seqNum} to retrieve a particular topic message by its sequence number from a topic. These APIs are considered alpha and may changed or be removed in the future. We also dramatically increased test coverage for the REST APIs and squashed some bugs in the process.
For our GRPC API, we had to switch from R2DBC to Hibernate to reach the scale and stability that we needed. In doing so, we can now support a lot more concurrent subscribers at a higher throughput. It should also finally put to rest any stability concerns with the GRPC component.
There are a few breaking changes that we had to make. We now no longer write and store record or balance files to the filesystem after they are inserted into the database. If you still need these files, you can set hedera.mirror.parser.balance.keepFiles and hedera.mirror.parser.record.keepFiles to true.
Also, we moved the persist properties to be grouped under a new path. That is we moved options like hedera.mirror.parser.record.persistTransactionBytes to hedera.mirror.parser.record.persist.transactionBytes. Please update your local configuration accordingly.
Enhancements
- Bump version for v0.8.0 #634
- Add support for Requester Pays S3 buckets #622
- Bump version for v0.8.0-rc1 #620
- Increase rest api integration coverage to match api offerings 2 #617
- Topic message by timestamp REST API #616
- Support REST API retrieval of topic message by topic id and sequence #615
- Topic message by sequence number REST API #614
- Add /topic/message/:consensusTimestamp API #613
- Increase rest api integration coverage to match api offerings #612
- Update copyright to 2020 #607
- Make monitor API configurable in dashboard #606
- REST Integration tests improvements 2 replacement #603
- Remove GRPC monitor #602
- Rename parsed directory and persist properties #601
- Updated integrationDbOps.js with more detailed setup logic #598
- Configuration to delete parsed files #596
- Update RecordFileParser.loadRecordFile() to match design #587
- Make monitoring endpoint configurable in dashboard #584
- Remove GRPC monitor #583
- Switch from R2DBC to Hibernate #544
- REST API integration tests for all parameters #412
- Don't keep parsed files #259
Bug Fixes
- Fix NPE in Downloader #619
- NullPointerException in Downloader #611
- Added logic to fail if all missing messages were not retrieved during a subscription #610
- Switch from R2DBC to Hibernate #597
- Subscribe topic should return error if no missing messages #589
- Stalled clients for historical data at higher page size #558
Contributors
We'd like to thank all the contributors who worked on this release!