Skip to content

Commit

Permalink
feat(indexer-example): add simple HTTP server for block data retrieval
Browse files Browse the repository at this point in the history
Implement a basic HTTP server within the indexer example, which allows for retrieving
block data through a new endpoint `/block/{height}`. This endpoint returns the block
data as JSON, making it easier to interact with and query block information.

The server utilizes the `actix-web` framework and integrates with the existing block
listening logic, providing a simple interface to access indexed block data within the
example application.
  • Loading branch information
AlexPBlack committed Jun 28, 2024
1 parent 7dd0b59 commit 70c80a5
Show file tree
Hide file tree
Showing 5 changed files with 107 additions and 265 deletions.
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion tools/indexer/example/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,13 @@ workspace = true

[dependencies]
actix.workspace = true
actix-web = "4"
serde = { version = "1.0", features = ["derive"] }
anyhow.workspace = true
clap.workspace = true
openssl-probe.workspace = true
serde_json.workspace = true
tokio.workspace = true
tracing.workspace = true

near-indexer.workspace = true
near-o11y.workspace = true
11 changes: 7 additions & 4 deletions tools/indexer/example/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
NEAR Indexer Simple Logger Example
==================================
/nearcore/tools/indexer/example

This is an example project featuring [NEAR Indexer Framework](https://github.com/nearprotocol/nearcore/tree/master/chain/indexer). This Indexer prints out all the blocks, chunks, transactions, receipts, execution outcomes, and state changes block by block immediately once it gets finalized in the network.
provide a api
curl -X GET http://127.0.0.1:6526/block/{number} | jq .
provide 10000 block, new block data will cover old data

Refer to the NEAR Indexer Framework README to learn how to run this example.
import { types } from 'near-lake-framework';

types.StreamerMessage
61 changes: 61 additions & 0 deletions tools/indexer/example/src/http_server.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
// http_server.rs

use actix_web::{web, App, Either, HttpResponse, HttpServer, Responder};
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use tokio::sync::mpsc;
use tracing::info;
use near_indexer;

pub(crate) type BlockCache = Arc<Mutex<HashMap<u64, near_indexer::StreamerMessage>>>;

pub async fn listen_blocks(mut stream: mpsc::Receiver<near_indexer::StreamerMessage>, cache: BlockCache) {
while let Some(streamer_message) = stream.recv().await {
let height = streamer_message.block.header.height;

{
let mut blocks = cache.lock().unwrap();
if blocks.len() >= 10000 {
let oldest_height = *blocks.keys().next().unwrap();
blocks.remove(&oldest_height);
}
blocks.insert(height, streamer_message.clone());
}

info!(
target: "indexer_example",
"#{} {} Shards: {}, Transactions: {}, Receipts: {}, ExecutionOutcomes: {}",
height,
streamer_message.block.header.hash,
streamer_message.shards.len(),
streamer_message.shards.iter().map(|shard| if let Some(chunk) = &shard.chunk { chunk.transactions.len() } else { 0usize }).sum::<usize>(),
streamer_message.shards.iter().map(|shard| if let Some(chunk) = &shard.chunk { chunk.receipts.len() } else { 0usize }).sum::<usize>(),
streamer_message.shards.iter().map(|shard| shard.receipt_execution_outcomes.len()).sum::<usize>(),
);
}
}

async fn get_block_by_height(data: web::Data<AppState>, height: web::Path<u64>) -> impl Responder {
let blocks = data.cache.lock().unwrap();
if let Some(block) = blocks.get(&height.into_inner()) {
Either::Left(web::Json(block.clone()))
} else {
Either::Right(HttpResponse::NotFound().body("Block not found"))
}
}

#[derive(Clone)]
pub struct AppState {
pub cache: BlockCache,
}

pub async fn run_server(cache: BlockCache) -> std::io::Result<()> {
HttpServer::new(move || {
App::new()
.app_data(web::Data::new(AppState { cache: cache.clone() }))
.route("/block/{height}", web::get().to(get_block_by_height))
})
.bind("127.0.0.1:6526")?
.run()
.await
}
Loading

0 comments on commit 70c80a5

Please sign in to comment.