How to run AWS nodes
Architecture Overview
This blueprint has two options for running nodes. You can set up a single JSON RPC node or multiple nodes in highly-available setup. The details are below.
Single RPC node setup
This setup is for small scale PoC or development environments. It deploys a single EC2 instance with both consensus and execution clients. The RPC port is exposed only to internal IP range of the VPC, while P2P ports allow external access to keep the clients synced.
Highly available setup
An ongoing data synchronization process is configured with nodes in the Ethereum network with a sync node and RPC nodes.
The sync node is used to create a copy of node's state data in Amazon S3 bucket.
When new RPC nodes are provisioned, they copy state data from Amazon S3 bucket to speed up the initial sync process.
Applications and smart contract development tools access highly available RPC nodes behind the Application Load Balancer.
Well-Architected
Solution Walkthrough
Open AWS CloudShell
To begin, ensure you login to your AWS account with permissions to create and modify resources in IAM, EC2, EBS, VPC, S3, KMS, and Secrets Manager.
From the AWS Management Console, open the AWS CloudShell, a web-based shell environment. If unfamiliar, review the 2-minute YouTube video for an overview and check out CloudShell with VPC environment that we'll use to test nodes API from internal IP address space.
Once ready, you can run the commands to deploy and test blueprints in the CloudShell.
Clone this repository and install dependencies
git clone https://github.com/aws-samples/aws-blockchain-node-runners.git
cd aws-blockchain-node-runners
npm installPrepare to deploy nodes
Make sure you are in the root directory of the cloned repository
If you have deleted or don't have the default VPC, create default VPC
aws ec2 create-default-vpcWith the Node Runners blueprints for Ethereum, you can deploy both single Ethereum nodes and multi-node high-availability configurations on AWS. Furthermore, Node Runners is designed to support client diversity, with configurations available for a variety of client combinations for the Execution Layer (EL) and Consensus Layer (CL).
Configure your setup.
Consensus Layer Client Options
To specify the Ethereum client combination you wish to deploy, create your own copy of .env file and edit it using your preferred text editor. The contents of your file for a Erigon / Lighthouse node deployment is as follows, which uses a sample config from the repository:
NOTE: You can find more examples inside the sample-configs directory, which illustrate other Ethereum client combinations.
Don’t see a client or client combination you would like supported? Open a GitHub issue or Pull Request, we encourage contribution to Node Runners!
Deploy common components such as IAM role, and Amazon S3 bucket to store data snapshots
pwd
# Make sure you are in aws-blockchain-node-runners/lib/ethereum
npx cdk deploy eth-commonOption 1: Single RPC Node
Deploy Single RPC Node
pwd
# Make sure you are in aws-blockchain-node-runners/lib/ethereum
npx cdk deploy eth-single-node --json --outputs-file single-node-deploy.jsonAfter starting the node you need to wait for the inital syncronization process to finish. It may take from half a day to about 6-10 days depending on the client combination and the state of the network. You can use Amazon CloudWatch to track the progress. There is a script that publishes CloudWatch metrics every 5 minutes, where you can watch
sync distancefor consensus client andblocks behindfor execution client. When the node is fully synced those two metrics shold show 0. To see them:Navigate to CloudWatch service (make sure you are in the region you have specified for
AWS_REGION)Open
Dashboardsand selecteth-sync-node-<your-eth-client-combination>from the list of dashboards.
Once the initial synchronization is done, you should be able to access the RPC API of that node from within the same VPC. The RPC port is not exposed to the Internet. Turn the following query against the private IP of the single RPC node you deployed:
INSTANCE_ID=$(cat single-node-deploy.json | jq -r '..|.node-instance-id? | select(. != null)')
NODE_INTERNAL_IP=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID --query 'Reservations[*].Instances[*].PrivateIpAddress' --output text)
echo "NODE_INTERNAL_IP=$NODE_INTERNAL_IP"Copy output from the last echo command with NODE_INTERNAL_IP=<internal_IP> and open CloudShell tab with VPC environment to access internal IP address space. Paste NODE_INTERNAL_IP=<internal_IP> into the new CloudShell tab. Then query the API:
# IMPORTANT: Run from CloudShell VPC environment tab
# We query token balance of Beacon deposit contract: https://etherscan.io/address/0x00000000219ab540356cbb839cbe05303d7705fa
curl http://$NODE_INTERNAL_IP:8545 -X POST -H "Content-Type: application/json" \
--data '{"method":"eth_getBalance","params":["0x00000000219ab540356cBB839Cbe05303d7705Fa", "latest"],"id":1,"jsonrpc":"2.0"}'The result should be like this (the actual balance might change):
{"jsonrpc":"2.0","id":1,"result":"0xe791d050f91d9949d344d"}Option 2: Highly Available RPC Nodes
Deploy Sync Node
pwd
# Make sure you are in aws-blockchain-node-runners/lib/ethereum
npx cdk deploy eth-sync-node --json --outputs-file sync-node-deploy.jsonAfter starting the node you need to wait for the inital syncronization process to finish. It may take from half a day to about 6-10 days depending on the client combination and the state of the network. You can use Amazon CloudWatch to track the progress. There is a script that publishes CloudWatch metrics every 5 minutes, where you can watch
sync distancefor consensus client andblocks behindfor execution client. When the node is fully synced those two metrics shold show 0. To see them:Navigate to CloudWatch service (make sure you are in the region you have specified for
AWS_REGION)Open
Dashboardsand selecteth-sync-node-<your-eth-client-combination>from the list of dashboards.
Once synchronization process is over, the script will automatically stop both clients and copy all the contents of the /data directory to your snapshot S3 bucket. That may take from 30 minutes to about 2 hours. During the process on the dashboard you will see lower CPU and RAM utilization but high data disc throughput and outbound network traffic. The script will automatically start the clients after the process is done.
Configure and deploy 2 RPC Nodes
pwd
# Make sure you are in aws-blockchain-node-runners/lib/ethereum
npx cdk deploy eth-rpc-nodes --json --outputs-file rpc-node-deploy.jsonGive the new RPC nodes about 30 minutes (up to 2 hours for Erigon) to initialize and then run the following query against the load balancer behind the RPC node created
export ETH_RPC_ABL_URL=$(cat rpc-node-deploy.json | jq -r '..|.alburl? | select(. != null)')
echo ETH_RPC_ABL_URL=$ETH_RPC_ABL_URL# IMPORTANT: Run from CloudShell VPC environment tab
# We query token balance of Beacon deposit contract: https://etherscan.io/address/0x00000000219ab540356cbb839cbe05303d7705fa
curl http://$ETH_RPC_ABL_URL:8545 -X POST -H "Content-Type: application/json" \
--data '{"method":"eth_getBalance","params":["0x00000000219ab540356cBB839Cbe05303d7705Fa", "latest"],"id":1,"jsonrpc":"2.0"}'The result should be like this (the actual balance might change):
{"jsonrpc":"2.0","id":1,"result":"0xe791d050f91d9949d344d"}If the nodes are still starting and catching up with the chain, you will see the following repsonse:
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
</body>Clearing up and undeploying everything
Destroy RPC Nodes, Sync Nodes and Common components
# Setting the AWS account id and region in case local .env file is lost
export AWS_ACCOUNT_ID=<your_target_AWS_account_id>
export AWS_REGION=<your_target_AWS_region>
pwd
# Make sure you are in aws-blockchain-node-runners/lib/ethereum
# Destroy Single RPC Node
cdk destroy eth-single-node
# Destroy RPC Nodes
cdk destroy eth-rpc-nodes
# Destroy Sync Node
cdk destroy eth-sync-node
# You need to manually delete an s3 bucket with a name similar to 'eth-snapshots-$accountid-eth-nodes-common' on the console,firstly empty the bucket,secondly delete the bucket,and then execute
# Delete all common components like IAM role and Security Group
cdk destroy eth-commonVisit the AWS Blockchain Node Runners page to learn more about the Node Runners project. If you have questions, you can ask them on AWS Re:Post, with a “blockchain” tag.
FAQ
How to check the logs of the clients running on my sync node?
NOTE: In this tutorial we chose not to use SSH and use Session Manager instead. That allows you to log all sessions in AWS CloudTrail to see who logged into the server and when. If you receive an error similar to
SessionManagerPlugin is not found, install Session Manager plugin for AWS CLI
pwd
# Make sure you are in aws-blockchain-node-runners/lib/ethereum
export INSTANCE_ID=$(cat sync-node-deploy.json | jq -r '..|.sync-node-instance-id? | select(. != null)')
echo "INSTANCE_ID=" $INSTANCE_ID
aws ssm start-session --target $INSTANCE_ID --region $AWS_REGION
sudo su ethereum
# Execution client logs:
docker logs --tail 50 execution -f
# Consensus client logs:
docker logs --tail 50 consensus -fHow to check the logs from the EC2 user-data script?
pwd
# Make sure you are in aws-blockchain-node-runners/lib/ethereum
export INSTANCE_ID=$(cat sync-node-deploy.json | jq -r '..|.sync-node-instance-id? | select(. != null)')
echo "INSTANCE_ID=" $INSTANCE_ID
aws ssm start-session --target $INSTANCE_ID --region $AWS_REGION
sudo cat /var/log/cloud-init-output.logI'm running sync node with Ethereum and Prysm or Lighthouse and it gets stuck during syncing, what should I do?
Usually restart helps Erigon client to re-connect with other nodes and continue syncing. To restart do the following:
pwd
# Make sure you are in aws-blockchain-node-runners/lib/ethereum
export INSTANCE_ID=$(cat sync-node-deploy.json | jq -r '..|.sync-node-instance-id? | select(. != null)')
echo "INSTANCE_ID=" $INSTANCE_ID
aws ssm start-session --target $INSTANCE_ID --region $AWS_REGION
sudo su ethereum
/usr/local/bin/docker-compose -f /home/ethereum/docker-compose.yml down
/usr/local/bin/docker-compose -f /home/ethereum/docker-compose.yml up -dResource links
Last updated