A. Joseph Dupre III
Technical Product Manager

OpenStack Clouds with Continuous Deployment Lifecycles Need Continuous Application Delivery Testing

August 5, 2016 by A. Joseph Dupre III

OpenStack is an open source project to move beyond working with virtualization in terms of individual servers or even groups of servers by providing a platform that can treat infrastructure as a pool of resources. The overall OpenStack is divided into projects that tackle the technical challenges required to make this happen. Nova is the codename for the application programming interface (API) that provides the compute resources. 

Nova allows virtual machines (VMs) to be targeted to resource pools, and can then execute them with the proper amount of CPU, Memory, and such. Nova allows for various hypervisors to be usable by OpenStack including kernel virtual machine (KVM) and VMware ESXi. Glance is the codename for the image management component of OpenStack. Glance is responsible for storing and retrieving images that need to be associated with a VM when its resources are provisioned for a compute resource. Storage service within OpenStack is handled by two projects whose codenames are Swift for object storage and Cinder for block storage.

A VM needs the compute, image, and storage services tied together with the virtual and physical networking that gives the VM connectivity to the world around it. The network service has been codenamed Neutron and implements an API and framework for interconnecting VMs, as well as the entire stack, to the physical world. Within the Neutron API, Layer-2 and Layer-3 devices can be modeled in a software defined network (SDN). These network layers can be highly varied between one OpenStack deployment and another.


Figure 1. OpenStack Cloud Operating System Framework (Source: OpenStack.org)

Challenges of Testing QoE within OpenStack-Based Clouds

OpenStack may use OpenvSwitch (OVS) to establish Layer 2 connectivity within a compute resource as well as provide for distributed virtual switching across multiple compute resources. Overlay technologies such as VXLAN or GRE may be used to further segment network paths. Neutron plugins allow third-party vendors to implement software hooks into their own virtual and physical switching solutions. With OpenStack, you’ll find large variations of network configuration and vendor interconnectivity, coupled with the increased flexibility of deploying VM connectivity in a software-defined model. This means that testing the end-to-end quality of a network-based service within an OpenStack-based cloud is just as critical as the physical network infrastructure testing for data center interconnects.


Figure 2. OpenStack Cloud OpenvSwitch Networking Stack (Source: OpenStack.org)

As a cloud-based solution, OpenStack provides a scalable and elastic solution for instantiating end-to-end service chains that can be controlled programmatically to scale up to meet high demand such as a large number of users all accessing web services at high peak usage times. Continuous-integration, continuous-delivery practices highly leveraged today mean that those elastic network services will not only be scaled up and down to meet demand, but will also be routinely modified and respawned with new features, bug fixes, or inserted into longer, more complex service chains.

In this cloud-based scenario, there is a real challenge to understand the effects of the network infrastructure decisions on the quality of the services that are chained over that infrastructure. As noted above, the layers of physical and software networking components that comprise the overlays and underlays that make up an OpenStack deployment are highly varied and each consideration has a dynamic impact on the overall performance—often in a non-deterministic way such that the only way to understand how a service is going to behave is after it has already been placed into service.

Faster iterations of service revisions, coupled with continuous delivery into live production public and private clouds, makes testing a challenge. Test tools for service verification and application delivery validation must be inserted into these dynamic service chains. As part of an agile testing methodology, testing must leverage the benefits of cloud elasticity with the flexibility to place the test tools anywhere within the OpenStack deployment to isolate individual points of failure or identify degraded quality of experience (QoE). Layer 2 and Layer 3 virtual test tools are critical to understanding the large-brush-stroke categorizations of the overlays and the underlays that underpin the service. However, they must be augmented by test tools capable of quantifying the application delivery performance and QoE for Layer 4 through Layer 7 to understand the true scalability and upper limits on tripleplay voice, video, and data traffic.

Multiplay Service Emulations to Measure Real-Time Application QoE

Questions about the OpenStack environment that need to be answered for a given service include:

  • How many compute nodes are required to support 10Gbps sustained application traffic?
  • What is the performance impact of switching from a VLAN-based network to a VXLAN overlay on the end-to-end application?
  • What is the impact on QoE for voice and video communications when the OpenStack environment has scaled to 90% compute capacity and background traffic from other cloud services is contending for the same network bandwidth?

These and many more questions can only be answered definitely by inserting an application delivery test tool capable of repeatable results at measuring QoE, quantifying concurrent users, connection success rates, and performance metrics such as throughput and latency into the continuous delivery service chain.

Ixia’s IxLoad VE emulates web, video, voice, storage, VPN, wireless, infrastructure, and encapsulation/security protocols to recreate real-world IT scenarios. It delivers real-time QoE metrics during end-to-end testing of converged application delivery infrastructure and services.


Figure 3. IxLoad VE Virtual Load Module and Virtual Test Controller Deployed in OpenStack

IxLoad VE offers VMware ESX/ESXi and KVM hypervisor support and is officially supported for use in cloud deployments based on OpenStack Liberty release. Virtual Load Module test components may be provisioned through the OpenStack GUI codenamed Horizon or via OpenStack REST APIs. To facilitate integration into automation and CICD service chains, IxLoad VE supports REST APIs for deployment and test execution. A sample Heat Template is also available for automated stack creation and service deployment acceleration.