Amazon Web Services, VPC Guide for Illinois

How to use Virtual Private Cloud (VPC) features with your University of Illinois Amazon Web Services (AWS) account.

Overview

Amazon Virtual Private Cloud (VPC) is a logically isolated virtual network in the AWS cloud which is dedicated to your AWS account. A single AWS account may have several VPCs.

Each VPC belongs to a single Region and may contain multiple subnets; each subnet belongs to a single Availability Zone within that region, and has a single route table.

This page focuses on VPC usage recommendations which are specific to the University of Illinois. For general topics, please consult Amazon's documentation.

Do I need VPC?

Your AWS account automatically comes with a default VPC in each region (and a default public-facing subnet in each Availability Zone) which you can use for many purposes without needing to know anything about Amazon VPC.  This includes interacting with services like Shibboleth which are designed to be accessible from the public Internet.

If you don't have any special requirements for accessing private resources either on the campus network or in other AWS accounts besides your own, the default VPC may be all you need (in which case you may safely skip reading the rest of this page).

Independent vs Enterprise VPC

Technology Services describes two categories of VPCs (these are University of Illinois terms not used by Amazon):

  • Enterprise VPCs must be created in coordination with Technology Services, using private IPv4 space which is centrally allocated to ensure that it will not overlap with the campus network or with other Enterprise VPCs.  This makes it possible to enable some additional networking features (detailed below).

  • Independent VPCs are created freely on your own, using any private IPv4 space you choose. The default VPC is also an Independent VPC.

The following table summarizes the differences between Independent and Enterprise VPCs:

  Independent VPC
Enterprise VPC
Region any us-east-2 (Ohio) only
Private IPv4 space
any allocated by Technology Services
limited in size
may be managed in IPAM
Public-facing subnets
supported supported
Private-facing subnets
supported supported
Campus-facing subnets
not supported supported
VPC Peering to other Enterprise VPCs
(including Core Services VPCs)

not supported supported

Notes:

  • This table introduces several new terms that will be explained in the remaining sections of this page.
  • If desired, you may use Technology Services IPAM to manage the private IPv4 space for an Enterprise VPC (but not for an Independent VPC), i.e. creating Host records whose reverse-mapping DNS will be visible to University clients.
  • Managing DNS for your public IPv4 addresses within Amazon works the same way for both types of VPCs.  See Amazon Web Services, Authoritative DNS Guide for Illinois for more information on this.
  • A VPC Peering Connection enables your Enterprise VPC to communicate directly with another Enterprise VPC in the same region. (It is also possible on an ad-hoc basis to create a peering between two Independent VPCs if you were fortunate enough to choose non-overlapping IP space, but Independent VPCs may not peer with Enterprise VPCs.)
In general, only choose an Enterprise VPC when you require VPC Peering Connections and/or campus-facing subnets.  If you don't need these features, choose an Independent VPC.

Core Services VPCs

Core Services VPCs are special Enterprise VPCs maintained by Technology Services to provide direct access to the following frequently needed services without leaving the AWS cloud:

Your Enterprise VPC must have a VPC Peering Connection to a Core Services VPC in order to use these services.  Some additional configuration may also be required.

Subnet Types

Technology Services describes three categories of subnets (the first two are terms also used by Amazon, the third is specific to the University of Illinois):

  • Public-facing subnets enable bi-directional communication (i.e. acting as either a client or a server) with any host on the public Internet.  Interfaces on a public-facing subnet still use private IPv4 addresses internally, but the AWS Internet Gateway automatically maps each private IP to an associated public IP or Elastic IP using 1:1 Network Address Translation (NAT).

  • Private-facing subnets cannot communicate directly with the public Internet. However, you can add a NAT Gateway (attached to a public-facing subnet) to enable outbound-only access from the private-facing subnet to the Internet using many-to-one NAT (e.g. for accessing other AWS services or downloading external software updates).

  • Campus-facing subnets are special private-facing subnets equipped by Technology Services with a VPN connection which enables bi-directional communication (without NAT) to the University of Illinois campus network.

The figure below provides a simple illustration of the different communication paths for public-facing subnets and campus-facing subnets (private-facing subnets would look like campus-facing subnets, but without the direct path to campus):


More generally, the following table summarizes the communications possible (though not necessarily permitted) for each type of subnet:

  Public-facing subnet
Private-facing subnet
Campus-facing subnet
To / from private IP in same VPC
yes yes yes
To / from private IP in other VPC
yes
(requires VPC peering)
yes
(requires VPC peering)
yes
(requires VPC peering)
Outbound to Internet
(including any public server)
yes
(requires public IP or Elastic IP)
yes
(requires NAT Gateway)
yes
(requires NAT Gateway)
Inbound from Internet
(including campus clients)
to AWS public IP
yes
(requires public IP or Elastic IP)
no no
Outbound to campus host
(not publicly accessible)
no no yes
Inbound from campus host
to AWS private IP
no no yes

Notes:

  • "Outbound to Internet" includes any publicly accessible service, even if the server in question happens to be located on campus or in another AWS VPC.  For example, Shibboleth can be used from a public-facing subnet with no special networking considerations.
  • Campus hosts with outbound Internet connectivity may act as clients for a publicly accessible service located in your AWS VPC; for this purpose they count as "Inbound from Internet".
  • Security Groups and Network ACLs may further restrict which of the possible communications are actually permitted to take place.

Detailed Example

The diagram below shows an Enterprise VPC with all three types of subnets duplicated across two Availability Zones, VPC Peering Connections to a Core Services VPC and one other Enterprise VPC, and also a VPC Endpoint for direct access to Amazon S3.  In practice, most Enterprise VPCs will not need all the elements shown here.


Notes:

  • The essential difference between the types of subnets is found in their associated route tables:
    • Public-facing subnets have a route table which includes a default route to the Internet via an Internet Gateway; campus-facing subnets and private-facing subnets do not (but may use a NAT Gateway instead).
    • Routes learned by the VPN Gateway are dynamically propagated into the route tables for campus-facing subnets only.  These routes include not just the Urbana-Champaign campus IP space (including RFC 1918 private IPv4 space) but also University-wide shared services and NCSA; see Guide to University of Illinois IP Spaces for details. 
  • We can choose which route tables will include the VPC Peering Connection and VPC Endpoint routes; in this case we have chosen to add them to all of our route tables, but this is not required.
  • The route to this VPC (10.x.y.0/24) is dynamically propagated from the VPN Gateway to routers on the University of Illinois campus network, but must be explicitly added to the desired route tables within the peer VPCs.
  • Each NAT Gateway has a network interface on a particular public-facing subnet and therefore depends upon that subnet's Availability Zone.  For full redundancy, our VPC therefore includes two NAT Gateways; the route tables shown for the campus-facing and private-facing subnets on the right use NAT Gateway 2 as their default route, while the campus-facing and private-facing subnets on the left have their own separate route tables (not shown) which use NAT Gateway 1 instead.  Both public-facing subnets use the same route table.
  • The Internet Gateway, VPN Gateway, VPC Peering Connections, and VPC Endpoint are not tied to a specific Availability Zone, so only one of each is needed.

Here are some concrete examples of possible (though not necessarily permitted) communications involving this VPC, with fabricated but plausible IP addresses:

  • An Internet host (198.51.100.17) can communicate bi-directionally with the public or Elastic IP (52.15.99.99) associated with instance A.  This public IP is translated by the AWS Internet Gateway to the private IP of instance A (10.x.y.40).
  • A campus Wi-Fi client (10.192.17.17) can initiate a connection to the public or Elastic IP (52.15.99.99) associated with instance A.  The source IP is translated by the campus exit firewall to a University of Illinois public IP (130.126.255.17), and the destination IP is translated by the AWS Internet Gateway to the private IP of instance A (10.x.y.40).
    View Illustration
  • A campus Wi-Fi client (10.192.17.17) can communicate bi-directionally with the private IP of instance B (10.x.y.100).  This communication traverses the VPN tunnel between AWS and campus, and does not involve NAT.
  • Instance B (10.x.y.100) or instance C (10.x.y.170) can initiate a connection to an Internet host (198.51.100.17).  The source IP is translated by NAT Gateway 2 to the NAT Gateway's own private IP (10.x.y.39) and then translated again by the AWS Internet Gateway to the Elastic IP (52.15.87.87) associated with NAT Gateway 2.
  • Any instance in this VPC (10.x.y.40, 10.x.y.70, 10.x.y.100) can initiate a connection to Amazon S3 (s3.us-east-2.amazonaws.com).  This communication uses the VPC Endpoint (because the destination IP matches the prefix list referenced in the route table), and does not involve the NAT Gateway.  Note that if there were no VPC Endpoint present, instances B and C could use the NAT Gateway to connect to S3 instead (and instance A could use the Internet Gateway).
  • Any instance in this VPC (10.x.y.40, 10.x.y.70, 10.x.y.100) can communicate bi-directionally with an instance in Core Services VPC n (10.224.n.17) or an instance in Other Enterprise VPC (10.v.w.17).  This communication traverses a VPC Peering Connection, and does not involve NAT.
  • Any instance in this VPC can communicate bi-directionally with any other instance in this VPC.

How to Build Your Enterprise VPC

  1. Determine which Enterprise networking features you need:
    • VPC peering to a Core Services VPC?
    • VPN connections to support campus-facing subnets?
    • both?

  2. Determine how much private IPv4 space you need for the entire VPC: is a /24 (256 addresses) sufficient, or do you anticipate using more than that within the near future?  Things to consider:
    • Which types of subnets will you need?  Almost every VPC will include public-facing subnet(s), but campus-facing and private-facing subnets are optional.
    • Do you want to use two Availability Zones instead of one (recommended)?  If so, you will need pairs of subnets, as shown in the Detailed Example.
    • How large does each subnet need to be?  (They do not all have to be the same size, but they must all be powers of two)  See also VPCs and Subnets.
    • VPCs cannot be resized once built.  If you do someday need more space, the recommended procedure is to create a new larger VPC, migrate your applications, and then decommission the old VPC.
    • Subnets also cannot be resized once built, but you can destroy and rebuild individual subnets inside your VPC without having to destroy the whole VPC.

  3. Contact Technology Services at aws-support@illinois.edu to request an Enterprise VPC allocation.  Include the following information in your request:
    • your 12-digit AWS account number
    • how much private IPv4 space you need
    • a brief description of what the VPC will be used for (example: "production VPC for Department of Redundancy Department")
    • who should be listed as contacts (Primary, Backup, Administrative) and permissions (Change Contacts, DNS) in the Contacts Database model?

  4. Download the example Infrastructure-as-Code from https://github.com/cites-illinois/aws-enterprise-vpc/ and follow the README instructions there to customize and run it.  In the middle of this process, you'll contact Technology Services again to enable the chosen Enterprise networking features for your new VPC.
As always, please feel free to contact Technology Services if you have questions or need help with any step of this process.

Frequently Asked Questions

Q: How can my service answer inbound requests from Internet clients and access private resources on campus?

  • A1: Use an internet-facing Elastic Load Balancer (i.e. Application Load Balancer or Network Load Balancer) as the entry point for your service.  The load balancer itself must be associated with public-facing subnets, but it can route requests to target instances on campus-facing (and/or private-facing) subnets.

  • A2: Architect your service using multiple tiers of instances.  Place the front-end instance(s) which need to answer client requests on public-facing subnets, and the back-end instance(s) which need to access private resources on campus-facing subnets.

  • A3: As a last resort, it is possible to assign two network interfaces to the same EC2 instance, placing the primary interface on a campus-facing subnet and the secondary interface on a public-facing subnet.  Multihoming is not for the faint of heart; the operating system on your EC2 instance will probably not handle it correctly out of the box, so you will have to configure and troubleshoot routing at the host level (which is tricky and very easy to get slightly wrong) before you can move on to building your service.  We recommend avoiding this scenario if at all possible, but contact Technology Services if you need help.

Q: Why can't I just create a single "everywhere-facing" subnet whose route table uses both the Internet Gateway and the VPN Gateway?

  • A: This combination is not supported because it creates an asymmetry in routing and NAT behavior which effectively makes it impossible for hosts on the campus network to communicate with the public or Elastic IP associated with your instance.

    Example: suppose client 130.126.255.17 initiates a connection to Amazon public IP 52.15.88.88, which is translated by the AWS Internet Gateway to the private IP of Instance B (10.x.y.100) on such a subnet.  When the instance attempts to reply, the destination IP 130.126.255.17 will match the campus-facing route 130.126.0.0/16, so the reply packet will traverse the VPN tunnel with untranslated source address 10.x.y.100 instead of passing through the Internet Gateway and having its source address translated back to 52.15.88.88.

Q: What is the purpose of including private-facing subnets in a VPC?

  • A1: In a multi-tier service architecture, putting back-end resources on private-facing subnets guarantees that they will never be accessible (inbound) from the public Internet.  However, you can use Security Groups to achieve the same goal for individual instances on public-facing subnets, so the question of whether this is valuable in practice comes down to a design preference; some people regard private-facing subnets as a useful conceptual boundary and/or an extra layer of security (e.g. in case of accidental Security Group misconfiguration).

  • A2: The fact that all outbound connections from a private-facing subnet to the public Internet appear to come from a single predictable IPv4 address (the Elastic IP of the NAT Gateway) may have practical value for certain use cases. Similar predictability can of course be achieved for individual instances on public-facing subnets by explicitly assigning an Elastic IP to each one, but that tactic doesn't generalize easily to Auto Scaling deployments.

Q: What about IPv6?

  • A: Amazon recently began supporting IPv6 for EC2 instances running in a VPC, but the overall feature set (including some network infrastructure functions and other AWS services) has not yet achieved full parity, and the conceptual framework is different enough that we are deferring any recommendation regarding the use of IPv6 in Enterprise VPCs until the offering has had more time to mature.





Keywords:VPC, AWS, Amazon, Cloud, Subnet   Doc ID:71015
Owner:David Z.Group:University of Illinois Technology Services
Created:2017-02-24 10:40 CSTUpdated:2017-09-11 12:52 CST
Sites:University of Illinois Technology Services
Feedback:  1   0