Solution Summary

Amazon Web Services (AWS) entered the NET+ program in 2014 — undergoing the rigorous, community-led service-validation process which allows institutions to form a collective voice in working with industry cloud providers to customize and enhance their service for higher education institutions and the extended community. After the service validation provided several key service enhancements for higher education, Internet2 community practitioners formed a Service Advisory Board to address evolving data transfer and administrative issues that were barriers to more ubiquitous use of the service for institutions, researchers and application developers. 

Collaborators

• NET+ Service Validation Institutions
  • University of Virginia (Sponsor)
  • Cornell University
  • Indiana University
  • University of Chicago
  • University of Iowa
• Amazon Web Services
• DLT Solutions (Program Reseller)

Products & Services

• NET+ Amazon Web Services
• Internet2 Cloud Connect
• Internet2 Network
• InCommon Federation

Community Resources

• Internet2 NET+ Program
• NET+AWS Service Advisory Board

The Project

The ability for the research and education (R&E) community to leverage a collective voice and have regular, ongoing dialogue with leading cloud service providers for the specialized needs of the academic enterprise, has always been at the heart of the Internet2 NET+ program. Amazon Web Services (AWS) entered the NET+ program in 2014 — undergoing the rigorous, community-led NET+ Service Validation process that works with industry cloud providers to customize and enhance their service for higher education institutions and the extended community. But, additional needed enhancements emerged to remove data transfer and administrative barriers. As with every NET+ service, Internet2 community practitioners formed a Service Advisory Board to address ongoing enterprise and user needs with the service. Their initial priorities were addressing data transfer and administrative issues that were barriers to the more ubiquitous use of the service for institutions, researchers, and application developers.

The Problem

After the AWS service was validated in 2014, it immediately offered institutions many enhancements including security and accessibility reviews, integration of InCommon single sign-on, Internet2 network performance optimizations, and tailored business and legal terms.

But, as the pace of research continues to accelerate, researchers increasingly rely on cloud computing to drive breakthrough science at breakneck speeds and are no longer limited by the availability of computing resources. Further, today’s globally distributed scientific research requires access to data repositories from participating scientists worldwide. Data from experiments all over the world must be accessible to those who need to collaborate.

Realizing this trend, the NET+ AWS Service Advisory Board began to work with AWS to underscore the importance of removing the uncertainty experienced by researchers for use of their cloud, through data egress fees–which are charges associated with data transfer from AWS to the Internet. These fees can potentially add up to significant unfunded expenditures for many central IT units and researchers considering the use of public clouds for their workloads, data in/out charges, and are a barrier to scientific collaboration. 

Particularly, for researchers, these unpredictable fees could be particularly concerning if they are making their research data available to other researchers for download (as is often required by funding agencies). If higher-than-anticipated demand for their data occurs, data egress charges would surpass their budget allocation.

Moreover, for the Internet2 member community, the idea of bandwidth-based charges runs counter to the two decades of investment in building a high-capacity, high-performance R&E network. The Internet2 Network is designed to maintain a sufficiently high level of capacity such that data transport by researchers and internet developers are never constrained by the cost of bandwidth or contention for capacity. So, for Internet2 members, data egress charges were viewed as an impediment to the effective use of cloud services–which are key to developing new applications and workflows that can advance science and scholarship.

Also, institutions need to build applications quickly, and securely–leveraging the agility of the cloud for building applications in the data center, and for greater cost savings and scalability. While, at the same time, maintaining security and quality. Over time, these advantages attract many different users who are seeking this agility–but, setting up account structures, roles and access privileges across different teams and departments were very difficult to manage effectively and became barriers for use or massive administrative burdens. Institutions needed a better way to manage these needs at scale.

Additionally, many research workloads require extensive connectivity to multiple US AWS regions or more specialized capabilities like jumbo frames, which allow large datasets to be moved more quickly from one computing environment to another.

The Solution

Seeing the need to make it easier for researchers to use AWS and for enterprise architects to support them, the AWS Service Advisory Board pointed their collective eyes toward two significant enhancements that would drastically improve the service for researchers and the professionals who support them: 

• Remove constraints of data egress fees to drive breakthrough science with cloud computing;
• Manage a sprawling cloud environment

Over the course of the Service Validation effort, the participants advocated for all institutions to be able to use its cloud storage, computing, and database services with waived data egress fees for qualified researchers and academic customers. Upon reviewing existing usage of the service, the participants and AWS identified an expected threshold of usage under which an institution would not be charged for data egress. This threshold ‘waiver’ model became the standard for cloud agreements, with a goal to decrease real and perceptive barriers to cloud adoption. The NET+ AWS Service Advisory Board continues to advocate for additional enhancements to ease adoption.  

Also, the service advisory board was successful working with AWS to launch “AWS Organizations” for NET+ AWS subscribers–gaining a valuable new tool in managing AWS accounts sprawl on their campus–and providing institutions a sound framework for managing accounts and services like network segmentation and authentication services like organization units, service control policies, centralized management for security, monitoring, and networking.

In addition to the NET+ AWS offering, AWS has worked with Internet2 and regional infrastructure partners to provide private peering services and direct access to AWS Direct Connect through the Internet2 Cloud Connect service. Regional and state networks and subscribers to NET+ AWS gain access to over 300 gigabits per second of private peering capabilities with AWS through the Internet2 Network, in addition to resilient national interconnects for private AWS Direct Connect. Together, these advanced, resilient, secure, community-enabled services make it easier for researchers to access and use cloud resources.

The Result

The NET+ Service Advisory Board and AWS worked together to achieve important service enhancements to make it easier for scientists to use, and provide more effective service management for enterprise architects. The Service Advisory Board continues to ensure the NET+ AWS program maintains feature parity with newly released AWS features and products while providing additional enhanced functionality to make it easier to deploy and scale on campus. 

In March 2016, AWS announced it would help make cloud and HPC budgeting more predictable for scientists by offering a 15% maximum discount of total monthly spending on AWS services, which is several times the typical usage among research customers. And, no costs are incurred to upload data into AWS or move data between Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Compute Cloud (Amazon EC2).

Further, in February 2019, Internet2 announced that NET+ AWS subscribers cloud usage had propelled the NET+ AWS program to an additional tier of discounting the Service Advisory Board had negotiated, with subscribing accounts now receiving a 5% discount on all AWS spending.

And, with the implementation of AWS Organizations for NET+ AWS subscribers, institutions now have a sound framework for managing AWS accounts, allowing institutions to:

• simplify the account creation process, billing, validation, consolidating, grouping and roles;
• control how services are used within individual accounts; and
• ensure controlled access to HIPAA and other regulated environments.

Now, university infrastructures can better support and advance scholarship and collaborative science. Institutions can more easily adopt, access, and use the resilient, secure, community-enabled AWS service–helping to free institutions from the overhead of managing infrastructure, provisioning servers, and configuring networks. 

Researchers can more predictably and reliably use public clouds for their workloads and make their data available to other researchers without fears that data egress charges would surpass their budget allocation, and enabling greater collaboration to accelerate scientific breakthroughs.

Through the unified voice and collaboration of Internet2 members, AWS has now been integrated with capabilities that mirror the fundamental philosophies of the Internet2 community and cyberinfrastructure: high-capacity, high-performance connectivity that never constrains research and development collaboration. 

Together, these efforts and solutions are a testament of how the research and education community is able to work together with industry to solve shared technology challenges and provide new platforms to develop applications and workflows that can advance science and scholarship.

About Internet2 NET+ AWS and Service Advisory Board

The NET+ AWS program is managed by an Internet2 program manager with the support of the NET+ AWS Service Advisory Board. The NET+ AWS Service Advisory Board reviews and prioritizes community feature requests on a periodic basis. Feature requests may be submitted to netplus@internet2.edu and the Service Advisory Board can be contacted at aws-campus-advisory@internet2.edu.

About DLT Solutions

DLT accelerates public sector growth for technology companies in federal, state, local, education, and healthcare markets. As the premier government solutions aggregator, DLT provides industry-leading technology companies access to a robust network of partners, a broad portfolio of contract vehicles, and dedicated channel enablement services.