Arch. Diagram
Shovel is an open source tool for synchronizing Ethereum data to your Postgres database.
Own your blockchain data without vendor lock-in.
Learn more in the docs.
Shovel processes multiple chains and multiple integrations concurrently. It starts indexing the latest block right away and optionally indexes historical data in the background.
Shovel is configured using declarative JSON that maps the data you care about onto your Postgres tables. No need to write custom functions or subgraphs for the basics. Simple things are easy and complex things are possible.
{
"pg_url": "postgres:///shovel",
"eth_sources": [
{"name": "mainnet", "chain_id": 1, "url": "https://ethereum-rpc.publicnode.com"},
{"name": "sepolia", "chain_id": 11155111, "url": "https://ethereum-sepolia-rpc.publicnode.com"}
],
"integrations": [
{
"name": "tokens",
"enabled": true,
"sources": [{"name": "mainnet"}, {"name": "goerli"}],
"table": {
"name": "transfers",
"columns": [
{"name": "log_addr", "type": "bytea"},
{"name": "block_time", "type": "numeric"},
{"name": "f", "type": "bytea"},
{"name": "t", "type": "bytea"},
{"name": "v", "type": "numeric"}
]
},
"block": [
{"name": "block_time", "column": "block_time"},
{
"name": "log_addr",
"column": "log_addr",
"filter_op": "contains",
"filter_arg": ["a0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"]
}
],
"event": {
"name": "Transfer",
"type": "event",
"anonymous": false,
"inputs": [
{"indexed": true, "name": "from", "type": "address", "column": "f"},
{"indexed": true, "name": "to", "type": "address", "column": "t"},
{"indexed": false, "name": "value", "type": "uint256", "column": "v"}
]
}
}
]
}
shovel=# select * from transfers limit 1;
-[ RECORD 1 ]------------------------------------------
block_time | 1699912391
f | \x66a89e05525bc2d4de6973ed2f30015f8ac4f165
t | \x7e40bfd3d80e060cdd7b8970ec967eb8adf121f9
v | 227300000
ig_name | tokens
src_name | mainnet
block_num | 18565786
tx_idx | 17
log_idx | 0
abi_idx | 0
log_addr | \xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48
# assumes you have pg running
createdb shovel
# download demo config file
curl -LO https://raw.githubusercontent.com/indexsupply/code/main/cmd/shovel/demo.json
# download shovel
curl -LO https://indexsupply.net/bin/1.0/darwin/arm64/shovel
chmod +x shovel
./shovel -config demo.json
Q: Is this similar to The Graph?
A: Yes. But there is no token model for Index Supply and Shovel is much simpler
Q: Is this similar to Dune?
A: You could build something like Dune with Shovel. But Shovel is like an OLTP system whereas Dune is an OLAP system. You wouldn’t want to build a UI using Dune as your backend.
Q: How many and what kind of chains are compatible with Shovel?
A: Any chain that uses the Ethereum JSON RPC API and ABI encoding. You can easily index 10 different chains with a single Shovel instance. Maybe more.