Custom RPCs

nodes/custom-rpc

runtimes/api-runtime

Remote Procedure Calls, or RPCs, are a way for an external program (eg. a frontend) to communicate with a Substrate node. They are used for checking storage values, submitting transactions, and querying the current consensus authorities. Substrate comes with several default RPCs. In many cases it is useful to add custom RPCs to your node. In this recipe, we will add three custom RPCs to our node. The first is trivial, the second calls into a custom runtime API, and the third interfaces with consensus.

The RPC Extensions Builder

In order to connect custom RPCs you must provide a function known as an "RPC extension builder". This function takes a parameter for whether the node should deny unsafe RPC calls, and returns an IoHandler that the node needs to create a json RPC. For context, read more at RpcExtensionBuilder trait API doc.

let rpc_extensions_builder = {
	let client = client.clone();
	let pool = transaction_pool.clone();
	Box::new(move |deny_unsafe, _| {
		let deps = crate::rpc::FullDeps {
			client: client.clone(),
			pool: pool.clone(),
			deny_unsafe,
			command_sink: command_sink.clone(),
		};

		crate::rpc::create_full(deps)
	})
};

This code is mostly boilerplate and can be reused. The one difference that you will encounter in your own node is the parameters that you pass. Here we've passed four parameters:

client - will be used in our second RPC
The transaction pool - we will not actually use it but many RPCs do
deny_unsafe - whether to deny unsafe calls
commands_sink - will be used in our third RPC

With this builder function out of the way we can begin attaching our actual RPC endpoints.

The Silly RPC

We'll begin by defining a simple RPC called "silly rpc" which just returns integers. A Hello world of sorts.

Defining the Silly RPC

Every RPC that the node will use must be defined in a trait. In the nodes/rpc-node/src/silly_rpc.rs file, we define a basic rpc as


#![allow(unused)]
fn main() {
#[rpc]
pub trait SillyRpc {
	#[rpc(name = "silly_seven")]
	fn silly_7(&self) -> Result<u64>;

	#[rpc(name = "silly_double")]
	fn silly_double(&self, val: u64) -> Result<u64>;
}
}

This definition defines two RPC methods called silly_seven and silly_double. Each RPC method must take a &self reference and must return a Result. Next, we define a struct that implements this trait.


#![allow(unused)]
fn main() {
pub struct Silly;

impl SillyRpc for Silly {
	fn silly_7(&self) -> Result<u64> {
		Ok(7)
	}

	fn silly_double(&self, val: u64) -> Result<u64> {
		Ok(2 * val)
	}
}
}

Finally, to make the contents of this new file usable, we need to add a line in our main.rs.


#![allow(unused)]
fn main() {
mod silly_rpc;
}

Including the Silly RPC

With our RPC written, we're ready to extend our IoHandler with it. We begin with a few dependencies in our rpc-node's Cargo.toml.

jsonrpc-core = "15.0"
jsonrpc-core-client = "15.0"
jsonrpc-derive = "15.0"
sc-rpc = '3.0'

Now we're ready to write the create_full function we referenced from our service. The function is quoted in its entirety below. This code is taken from nodes/rpc-node/src/rpc.rs.

pub fn create_full<C, P>(
	deps: FullDeps<C, P>,
) -> jsonrpc_core::IoHandler<sc_rpc::Metadata> where
	// --snip--
{

	let mut io = jsonrpc_core::IoHandler::default();

	// Add a silly RPC that returns constant values
	io.extend_with(crate::silly_rpc::SillyRpc::to_delegate(
		crate::silly_rpc::Silly {},
	));

	// --snip--

	io
}

These few lines extend our node with the Silly RPC.

Calling the Silly RPC

Once your node is running, you can test the RPC by calling it with any client that speaks json RPC. One widely available option is curl.

$ curl http://localhost:9933 -H "Content-Type:application/json;charset=utf-8" -d   '{
     "jsonrpc":"2.0",
      "id":1,
      "method":"silly_seven",
      "params": []
    }'

To which the RPC responds

{"jsonrpc":"2.0","result":7,"id":1}

You may have noticed that our second RPC takes a parameter, the value to double. You can supply this parameter by including its in the params list. For example:

$ curl http://localhost:9933 -H "Content-Type:application/json;charset=utf-8" -d   '{
     "jsonrpc":"2.0",
      "id":1,
      "method":"silly_double",
      "params": [7]
    }'

To which the RPC responds with the doubled parameter

{"jsonrpc":"2.0","result":14,"id":1}

RPC to Call a Runtime API

The silly RPC demonstrates the fundamentals of working with RPCs in Substrate. Nonetheless, most RPCs will go beyond what we've learned so far and actually interacts with other parts of the node. In this second example, we will include an RPC that calls into the sum-storage runtime API from the runtime API recipe. While it isn't strictly necessary to understand what the runtime API does, reading that recipe may provide helpful context.

Defining the Sum Storage RPC

Because this RPC's behavior is closely related to a specific pallet, we've chosen to define the RPC in the pallet's directory. In this case the RPC is defined in pallets/sum-storage/rpc. So rather than using the mod keyword as we did before, we must include this RPC definition in the node's Cargo.toml file.

sum-storage-rpc = { path = "../../pallets/sum-storage/rpc" }

Defining the RPC interface is similar to before, but there are a few differences worth noting. First, the struct that implements the RPC needs a reference to the client. This is necessary so we can actually call into the runtime. Second the struct is generic over the BlockHash type. This is because it will call a runtime API, and runtime APIs must always be called at a specific block.


#![allow(unused)]
fn main() {
#[rpc]
pub trait SumStorageApi<BlockHash> {
	#[rpc(name = "sumStorage_getSum")]
	fn get_sum(
		&self,
		at: Option<BlockHash>
	) -> Result<u32>;
}

/// A struct that implements the `SumStorageApi`.
pub struct SumStorage<C, M> {
	client: Arc<C>,
	_marker: std::marker::PhantomData<M>,
}

impl<C, M> SumStorage<C, M> {
	/// Create new `SumStorage` instance with the given reference to the client.
	pub fn new(client: Arc<C>) -> Self {
		Self { client, _marker: Default::default() }
	}
}
}

The RPC's implementation is also similar to before. The additional syntax here is related to calling the runtime at a specific block, as well as ensuring that the runtime we're calling actually has the correct runtime API available.


#![allow(unused)]
fn main() {
impl<C, Block> SumStorageApi<<Block as BlockT>::Hash>
	for SumStorage<C, Block>
where
	Block: BlockT,
	C: Send + Sync + 'static,
	C: ProvideRuntimeApi,
	C: HeaderBackend<Block>,
	C::Api: SumStorageRuntimeApi<Block>,
{
	fn get_sum(
		&self,
		at: Option<<Block as BlockT>::Hash>
	) -> Result<u32> {

		let api = self.client.runtime_api();
		let at = BlockId::hash(at.unwrap_or_else(||
			// If the block hash is not supplied assume the best block.
			self.client.info().best_hash
		));

		let runtime_api_result = api.get_sum(&at);
		runtime_api_result.map_err(|e| RpcError {
			code: ErrorCode::ServerError(9876), // No real reason for this value
			message: "Something wrong".into(),
			data: Some(format!("{:?}", e).into()),
		})
	}
}
}

Installing the Sum Storage RPC

To install this RPC , we expand the existing create_full function from rpc.rs.


#![allow(unused)]
fn main() {
io.extend_with(sum_storage_rpc::SumStorageApi::to_delegate(
	sum_storage_rpc::SumStorage::new(client),
));
}

Using RPC Parameters

This RPC takes a parameter ,at, whose type is Option<_>. We may call this RPC by omitting the optional parameter entirely. In this case the implementation provides a default value of the best block.

$ curl http://localhost:9933 -H "Content-Type:application/json;charset=utf-8" -d   '{
     "jsonrpc":"2.0",
      "id":1,
      "method":"sumStorage_getSum",
      "params": []
    }'

We may also call the RPC by providing a block hash. One easy way to get a block hash to test this call is by copying it from the logs of a running node.

$ curl http://localhost:9933 -H "Content-Type:application/json;charset=utf-8" -d   '{
     "jsonrpc":"2.0",
      "id":1,
      "method":"sumStorage_getSum",
      "params": ["0x87b2e4b93e74d2f06a0bde8de78c9e2a9823ce559eb5e3c4710de40a1c1071ac"]
    }'

As an exercise, change the storage values and confirm that the RPC provides the correct updated sum. Then call the RPC at an old block and confirm you get the old sum.

Polkadot JS API

Many frontends interact with Substrate nodes through Polkadot JS API. While the Recipes does not strive to document that project, we have included a snippet of javascript for interacting with these first two custom RPCs in the nodes/rpc-node/js directory.

The Manual Seal RPC

Our third and final example RPC will interact with consensus. Specifically, it will tell the consensus engine when to author and finalize blocks. The API for this RPC if defined in Substrate in the ManualSealApi Trait.

Installing the Manual Seal RPC

The previous RPC needed a reference to the client to call into the runtime. Likewise, this RPC needs a command stream to send messages to the actual consensus engine. This recipe does not cover installing the manual seal engine, but it is nearly identical to the instant seal engine used in the Kitchen Node.

To install the RPC endpoint, we do exactly as we have before, and extend the create_full function in rpc.rs

io.extend_with(
	// We provide the rpc handler with the sending end of the channel to allow the rpc
	// send EngineCommands to the background block authorship task.
	ManualSealApi::to_delegate(ManualSeal::new(command_sink)),
);

Using Manual Seal

Once your node is running, you will see that it just sits there idly. It will accept transactions to the pool, but it will not author blocks on its own. In manual seal, the node does not author a block until we explicitly tell it to. We can tell it to author a block by calling the engine_createBlock RPC.

The easiest way is to use Apps's Developer -> RPC Calls tab.

Calling the createBlock endpoint via Apps

It can also be called using curl as described previously.

$ curl http://localhost:9933 -H "Content-Type:application/json;charset=utf-8" -d   '{
     "jsonrpc":"2.0",
      "id":1,
      "method":"engine_createBlock",
      "params": [true, false, null]
    }'

This call takes three parameters, each of which are worth exploring.

Create Empty

create_empty is a Boolean value indicating whether empty blocks may be created. Setting create-empty to true does not mean that an empty block will necessarily be created. Rather it means that the engine should go ahead creating a block even if no transaction are present. If transactions are present in the queue, they will be included regardless of create_empty's value.'

Finalize

finalize is a Boolean indicating whether the block (and its ancestors, recursively) should be finalized after creation. Manually controlling finality is interesting, but also dangerous. If you attempt to author and finalize a block that does not build on the best finalized chain, the block will not be imported. If you finalize one block in one node, and a conflicting block in another node, you will cause a safety violation when the nodes synchronize.

Parent Hash

parent_hash is an optional hash of a block to use as a parent. To set the parent, use the format "0x0e0626477621754200486f323e3858cd5f28fcbe52c69b2581aecb622e384764". To omit the parent, use null. When the parent is omitted the block is built on the current best block. Manually specifying the parent is useful for constructing fork scenarios and demonstrating chain reorganizations.

Manually Finalizing Blocks

In addition to finalizing blocks while creating them, they can be finalized later by using the second provided RPC call, engine_finalizeBlock.

$ curl http://localhost:9933 -H "Content-Type:application/json;charset=utf-8" -d   '{
     "jsonrpc":"2.0",
      "id":1,
      "method":"engine_finalizeBlock",
      "params": ["0x0e0626477621754200486f323e3858cd5f28fcbe52c69b2581aecb622e384764", null]
    }'

The two parameters are:

The hash of the block to finalize.
A Justification. TODO what is the justification and why might I want to use it?

Substrate Recipes