Skip to main content

Step 2: Create the Database Infrastructure

What This Step Does​

In this step you build three things inside your AWS account:

  1. Networking — a private network (VPC) where Neptune will live, plus a gateway so Neptune can reach your S3 bucket.
  2. Database — an Amazon Neptune cluster with one database instance.
  3. Permissions — an IAM role that lets Neptune read data files from S3.

After this step, Neptune will be online and ready to receive data.

Why does Neptune need its own network?​

Neptune is designed to be private — it is not exposed to the public internet. This is a security feature. The downside is that you cannot connect to it directly from your laptop. In Step 3 you will create a small helper machine (EC2 instance) inside the same network to run commands against Neptune.

Shortcut: Automated Scripts​

If you prefer not to click through the AWS Console, you can run two scripts that create everything automatically:

export AWS_PROFILE=cloudbank-demo-admin
set -a && source .env && set +a

# 1. Create the IAM role that lets Neptune read from S3
bash scripts/06_create_neptune_load_role.sh

# 2. Create the VPC networking, Neptune cluster, and instance.
#    This script waits until the instance is Available (5-15 minutes),
#    then attaches the IAM role to the cluster and verifies the result.
bash scripts/07_create_neptune_cluster.sh

# 3. Confirm the database instance is ready (returns immediately after step 2)
aws neptune wait db-instance-available \
  --db-instance-identifier cloudbank-biobricks-neptune-1 \
  --region us-west-2

These scripts auto-detect your default VPC and subnets, create the S3 gateway endpoint, and write NEPTUNE_ENDPOINT and NEPTUNE_IAM_ROLE_ARN into your .env file.

The scripts create the Neptune instance with class db.r5.large by default (the class recommended for this dataset, see the Goal section below). If you want a cheaper smaller class anyway, override it: NEPTUNE_INSTANCE_CLASS=db.t3.medium bash scripts/07_create_neptune_cluster.sh.

scripts/07_create_neptune_cluster.sh prints an explicit result for the role association: either Role is associated to cluster or a warning with the exact command to re-run if the role was not attached yet (this only happens rarely, when IAM propagation lags the cluster readiness).

If you use the scripts, skip to Section 7 — Verify to confirm it worked.

Goal​

By the end of this step you have:

ResourceValue
Neptune clustercloudbank-biobricks-neptune in us-west-2
Database instancedb.r5.large (2 vCPU, 16 GiB RAM), status = Available
Security groupAllows inbound TCP on port 8182 from your private network
IAM roleNeptuneLoadFromS3Role with S3 read access
.env updatedNEPTUNE_ENDPOINT and NEPTUNE_IAM_ROLE_ARN filled in

Why db.r5.large? The full OREGANO file is large (about 114 MB, over 3 million triples), and a small instance class can run out of memory during the bulk load in Step 3, failing with OUT_OF_MEMORY_ERROR before the data is imported. db.r5.large (2 vCPU, 16 GiB) has 4 times the memory of db.t3.medium and loads the file reliably in about 10 minutes. It costs a few tens of cents per hour while it is running, and only for the hours the database exists (see the Amazon Neptune pricing page for the exact rate in your region). Smaller classes such as db.t3.medium or db.t4g.medium may still work, but they can hit out-of-memory on the full dataset.

Permission Note​

The regular cloudbank-demo profile may not have enough permissions to create VPCs, Neptune clusters, or IAM roles. If you see AccessDenied or UnauthorizedOperation, switch to the admin profile:

export AWS_PROFILE=cloudbank-demo-admin

Manual Console Walkthrough​

Use this section if you want to understand each piece, or if the automated scripts do not work in your environment.

1) Set Up Networking (VPC)​

You can use an existing default VPC or create a new one. Neptune needs:

  • At least 2 private subnets in different Availability Zones (AZs).
  • An S3 Gateway endpoint so Neptune can read from S3 without going through the public internet.

In the AWS Console:​

  1. Open the VPC service in us-west-2.
  2. Select or create a VPC.
  3. Make sure it has at least two subnets in different AZs.
  4. Create an S3 endpoint:
    • Go to Endpoints → Create endpoint.
    • Service: com.amazonaws.us-west-2.s3
    • Type: Gateway
    • Attach the route tables used by your subnets.

Why two subnets? Neptune requires a "DB subnet group" that spans multiple AZs for availability, even though this demo only runs one instance.

2) Create a Neptune Subnet Group​

A subnet group tells Neptune which subnets it is allowed to use.

  1. Open Amazon Neptune → Subnet groups → Create DB subnet group.
  2. Name: neptune-demo-subnet-group.
  3. Select your VPC.
  4. Add at least two private subnets in different AZs.

3) Create a Security Group​

A security group acts as a firewall. You need one that allows traffic on port 8182 (Neptune's port).

  1. Open EC2 → Security Groups → Create security group.
  2. Name: neptune-demo-sg.
  3. VPC: same VPC as above.
  4. Add an inbound rule:
    • Type: Custom TCP
    • Port: 8182
    • Source: your VPC's CIDR range (this is what the automated scripts in the Shortcut section create).
  5. Leave outbound rules as the default (allow all).

Tip: In Step 3 the EC2 runner is placed in this same security group, so it can already reach Neptune on port 8182. The VPC CIDR source is fine for this demo; a security-group source is more secure if you want to limit access to specific machines.

4) Create the Neptune Cluster and Instance​

  1. Open Amazon Neptune → Databases → Create database.
  2. Engine: Neptune.
  3. Cluster identifier: cloudbank-biobricks-neptune.
  4. Instance class: db.r5.large (2 vCPU, 16 GiB). The smaller db.t3.medium and db.t4g.medium classes are cheaper but can run out of memory during the bulk load in Step 3, so this demo uses db.r5.large.
  5. DB subnet group: neptune-demo-subnet-group.
  6. VPC security group: neptune-demo-sg.
  7. IAM DB authentication: Enable.
  8. Public access: Disable (recommended).
  9. Click Create database.

Wait until the status changes from creating to Available. This typically takes 5–15 minutes.

5) Create an IAM Role for the Bulk Loader​

Neptune does not use your credentials to read from S3. Instead, it assumes an IAM role that you create. This role grants Neptune read-only access to your S3 bucket.

  1. Open IAM → Roles → Create role.
  2. Trusted entity type: AWS service.
  3. Service: RDS (Neptune uses the RDS trust path internally).
  4. Attach policy: AmazonS3ReadOnlyAccess.
  5. Role name: NeptuneLoadFromS3Role.
  6. Click Create role and copy the role's ARN (e.g., arn:aws:iam::123456789012:role/NeptuneLoadFromS3Role).

Then attach the role to the cluster. Do this only after the cluster status is Available; adding it while the cluster is still creating fails silently:

aws neptune add-role-to-db-cluster \
  --region us-west-2 \
  --db-cluster-identifier cloudbank-biobricks-neptune \
  --role-arn arn:aws:iam::<your-account-id>:role/NeptuneLoadFromS3Role

6) Update Your .env File​

Add the two new values:

NEPTUNE_ENDPOINT=<your-cluster-endpoint>
NEPTUNE_IAM_ROLE_ARN=arn:aws:iam::<your-account-id>:role/NeptuneLoadFromS3Role

Then reload:

set -a
source .env
set +a

7) Verify Everything Is Ready​

Run these two commands to confirm the cluster and role exist:

# Print the Neptune endpoint
aws neptune describe-db-clusters \
  --region us-west-2 \
  --query "DBClusters[?DBClusterIdentifier=='cloudbank-biobricks-neptune'].Endpoint" \
  --output text

# Print the IAM role ARN
aws iam get-role \
  --role-name NeptuneLoadFromS3Role \
  --query "Role.Arn" \
  --output text

If both return values, you are ready for Step 3.

Check the role is attached to the cluster​

The bulk loader uses the role only when it is attached to the cluster. The shortcut script does this automatically after the instance is Available; on the manual path you ran the attachment command in Section 5:

aws neptune describe-db-clusters \
  --region us-west-2 \
  --db-cluster-identifier cloudbank-biobricks-neptune \
  --query 'DBClusters[0].AssociatedRoles' \
  --output text

If this prints your role ARN, the load role is attached. If it prints nothing, run the attachment command from Section 5 again (the cluster status must be Available).

Check the TLS certificate is valid (pre-check)​

Neptune serves a TLS certificate on port 8182. On rare occasions a brand-new cluster is assigned a certificate whose validity period has already ended, and then every TLS connection fails with certificate has expired (the bulk loader, curl, and Python clients are all affected). There is no console option or API to rotate the certificate, so it is worth checking before the first load.

The endpoint is only reachable from inside the VPC, so you run this command on the small EC2 runner that Step 3 creates, right before the bulk load (Step 3 shows how to send it to the runner):

openssl s_client -connect $NEPTUNE_ENDPOINT:8182 -servername $NEPTUNE_ENDPOINT -showcerts </dev/null 2>/dev/null | openssl x509 -noout -dates

The output looks like:

notBefore=Jul 27 00:00:00 2026 GMT
notAfter=Feb  9 23:59:59 2027 GMT

If notAfter is in the past (or the command reports certificate has expired), see Step 3 Troubleshooting: delete and recreate the cluster, or contact AWS support. With a valid certificate, the documented load and query steps work without any changes.

Troubleshooting​

ProblemLikely causeFix
AccessDenied when creating resourcesWrong profileRun export AWS_PROFILE=cloudbank-demo-admin
Neptune endpoint is emptyCluster is still provisioningWait a few minutes and re-run the describe command
IAM role ARN missingRole was not createdGo back to Section 5 and create it
Loader later says "can't find credential for iam_role_arn"Role is not attached to Neptune or the STS endpoint is missing in the VPCAttach the role while the cluster is Available: aws neptune add-role-to-db-cluster ... (verify with the check in Section 7) and create an STS interface endpoint (com.amazonaws.us-west-2.sts) in the VPC