4 Things We Learned From a Community Test Drive of the Internet2 Insight Console at TechEX

By Christopher Green - Digital Designer and Software Architect

We’re not testing you; you’re helping us test the software. So you can’t do anything wrong! 

With that disclaimer setting the stage for unfettered clicking and unfiltered feedback, we invited community members to take the new Internet2 Insight Console for a test drive at the 2022 Technology Exchange. And the community’s engagement in the process far exceeded our expectations. 

Altogether, we conducted 13 usability tests of the Insight Console in two days. More than 15 community members directly participated in the tests, some solo and others teaming up with colleagues. Many more attendees engaged indirectly, stopping by to observe and cheer on their peers.

Why and How of Usability Testing

The Internet2 Network Services Infrastructure Systems and Software (ISS) team has shared in-depth about what the Insight Console enables for the research and education community and how the community plays an essential role in our process to design and develop it. Since the very beginning of our work on the console in 2021, we’ve engaged the Internet2 member community in user research, UX design, UI prototyping, and usability testing.

The usability testing we conducted at TechEX was focused specifically on the Virtual Networks / Cloud Connect service within the console. That service will enhance and extend the existing capabilities of the popular OESS and Cloud Connect Portal. 

We had three goals for the usability tests:

1. We wanted to test some of our design assumptions, including a new visual representation of virtual networks used in place of plain text data.
2. We wanted to gauge whether the language and key concepts used in the console would resonate with the user community.
3. We wanted to share with the community more about Internet2’s usability testing, which is unique in the network engineering space. 

We asked usability testers to complete common tasks using a test version of the console. In one scenario, we asked them to create a Layer 3 VPN linking a Microsoft Azure ExpressRoute service to Columbia University. We also invited them to think out loud while completing the task (a technique known as the “concurrent think-aloud protocol”) to uncover pain points and imagine better ways of doing things while the experience is fresh – and the response is unfiltered.

We mirrored the live testing on a large screen in the exhibit foyer at TechEX, which made it possible for everyone at the conference to be exposed to the Insight Console and the usability testing process. Many attendees stopped by to watch. Some of them later signed up to participate in the testing, too. All to say, we were simultaneously conducting and demoing usability testing for the community.

What We Learned

Usability testing is rarely done in the network engineering space. Engaging Internet2 members in this process is a differentiator – one that may set the Internet2 Insight Console apart from the other tools that engineers and others in our community encounter daily.

Here are four things we took away from the usability testing at TechEX:

1. Community members love the look and feel of the Insight Console overall, and they appreciate that the design is clean, clear, and consistent across multiple services and tools.
2. Some of the terminology used in the console needs to be adjusted – whether it’s unfamiliar or outright confusing. One example is our use of the term “canvas,” which needs to be created as a first step in building a new virtual network. The term doesn’t resonate with the community, so we’re swapping it out for a more familiar term, asking them to first “create a new space.”
3. The visual diagram used to represent virtual networks is preferred over blocks of plain text in the Virtual Networks / Cloud Connect service, but we need to re-think the diagram to better fit the community’s experience and perspective. For example, the original curved shape of the diagram didn’t work; testers challenged us to come up with a new design that displays information more horizontally and linearly.
4. Community members, especially network engineers, want to see the details. For example, within that same diagram, we originally kept the visual representation of the path very simple (only showing the major hops), but usability testing revealed that we need to put the details back into the diagram. Effectively displaying the details for a BGP peering session is another challenge that we want to improve within the console, too. Usability testing shows that we’re on the right track.

Since TechEX, we’ve been working to analyze and respond to the feedback we gathered. We’ve entered all the screen recordings and audio transcripts from the usability tests into a central database, which allows us to search for keywords and conduct textual analysis to identify common pain points and areas for improvement.

This is an approach that we’ll use going forward – recording all usability testing sessions and then collecting that feedback in the database. It will allow us to build up our institutional knowledge as we plan for the future of the Internet2 Insight Console.

Comments From the Community

“To be able to offer feedback at that point in the development of the Insight Console interface was very helpful. It offered a means to get familiar with the direction being taken while also being able to address some of the odd choices that always seem to pop up when systems people develop hardware interfaces. There are always one or two assumptions made that seem a bit off. It seems Christopher and his group have taken that into account and made a lot of effort towards useability and not nailing down design choices that don’t make sense in the end product. Their openness and willingness to use criticisms in a positive way gave me a very impressive hope for the final product. I am excited to see it.” – Shannon Spurling, MOREnet

“As a recent statenet user who frequently used the Router Proxy to troubleshoot issues, I really love that the new Internet2 Insight Console allows a command to be run on several nodes simultaneously. This makes troubleshooting a problem like ‘what does this route look like across the network’ more efficient. I appreciate the team’s willingness to show their work, take great effort to collect feedback, and rethink or adjust deliverables accordingly.” – Brad Fleming, Internet2

What’s Next

The Insight Console will soon offer the community access to Looking Glass, which will replace Internet2’s existing Router Proxy service. Development work is still in progress on Virtual Networks, which will replace and enhance OESS and the Cloud Connect Portal.

Internet2 staff and community members will share the latest updates on the Insight Console at the 2023 Community Exchange. Register to join us May 8-11 in Atlanta! 

ICYMI

• How Big a Deal is the Internet2 Insight Console?
• Community Is at the Center of ‘Getting It Right’ for the Internet2 Insight Console