How To Optimize Access to Your Monitoring Data
When it comes to monitoring your network, data collection is an extremely important subject. You need to know the type and quality of your data. For instance, is it an exact copy of the network data or has your monitoring data been modified (time stamps, checksums, etc.). As you would naturally assume, good data will produce good results whereas bad data won’t. There are two underlying questions though—how do actually get good data and how do you know it is good?
The solution is to implement a series of best practices. There are three best practices that stand out:
- Implement a Visibility Architecture
- Install the right data access solution
- Use passive access devices to collect data
These best practices will ensure that you meet your desired goal. I’ll elaborate further on them here but you can watch these podcasts instead if you like: Tap Best Practices, Taps Vs. SPANs, and Visibility Architectures - Best Practices for Network Monitoring.
Network visibility (monitoring) best practices start with a proper architecture. You need to know what you have in your network and what its purpose. There are three basic components to a visibility architecture – the access layer, control layer, and the traditional monitoring tool layer. The first layer of the model is the monitoring data access layer. This is provided by some sort of network access device, whether it’s a physical tap, virtual tap, or mirrored port. The next layer uses a network packet broker, also called an NPB, which allows you to filter, aggregate and load balance data. From a business point of view, there are several high level benefits that packet brokers provide. These include: connectivity, scaling, reliability, longevity, and reduced tools costs. The third layer of the architecture is the security and monitoring tools. This is probably what most people are familiar with. Instead of receiving the data directly from a tap, these tools now receive filtered data from the NPB that is more relevant and concise. NPBs are designed to augment your monitoring tools, not replace them, which allows the tools to be more efficient.
The following diagram illustrates a generic visibility architecture:
The next item to consider is that the quality of your monitoring data matters. In this case, data quality is synonymous with the source of the data because the source of the data can effect analysis functions like troubleshooting and network security. For data collection, there are four common sources of network access to the data.
- Taps – These are passive devices that make a copy of the data that flows through them. While they are technically installed inline in the network, they only make a copy of the data, i.e. they don’t divert the main traffic flow. Neither the copied data nor the live network data are modified by this device. The tap simply replicates the data and sends that data out a monitoring port to the NPB. These devices can be located anywhere within the network and once they are installed, you don’t have to disrupt the network again. All of the data is forwarded to the NPB for filtering and routing to security and monitoring tools.
- Bypass switch – This device is similar to a tap except that it directly diverts the live network data, shunting it off to a network packet broker where the packets can be sent to specialized tools (e.g. firewall, IPS, next gen firewall, WAF, etc.) for analysis. These devices are typically located at the ingress/egress for the network. They have a special bypass function that can be automatically engaged should there be a device failure for any tools connected to the bypass switch. This provides fail-over mechanisms for network survivability. Essentially, should the security tool go offline, traffic can be automatically allowed to pass downstream instead of causing a network failure (potentially resulting in significant monetary loss).
- Virtual taps – These are similar to a standard tap except that they are software products designed for the virtual data center and or cloud environments. These virtual taps give you access to east-west data from virtual servers that you typically would not have access to with a traditional physical tap.
- SPAN ports – This is active technology that is incorporated into many network data switches. These ports allow you to selectively mirror traffic from the switch to a port that can be connected to an NPB or tool. These ports are part of network switches and are typically located in the core of the network. Since these ports use active technology, instead of passive technology like taps and bypass switches, they can alter the data packets so this needs to be understood if they are used.
The third part of the best practices mentioned above focuses on the use of passive devices. You definitely want passive devices. Active devices, like SPAN ports, can change timestamps, checksum, and header information that could be useful for troubleshooting activities. Here is a short list of reasons why taps are superior to SPAN ports. SPAN ports have the following issues:
- They create duplicated data packets that reduce the efficiency of your monitoring tools
- SPAN ports only provide summarized data
- SPANs don’t provide a complete copy. This means there is missing data (Layer 1 data, corrupted and malformed packets, and other data oddities) that is not forwarded through SPAN ports.
- They require CLI programming
- SPAN ports change the timestamps of packets
- SPAN ports have been shown to be hackable (so they can be a security risk)
I recommend using Taps as part of a well-planned visibility architecture. The passive technology means that you can trust the data. If SPAN ports are used, you’ll have to spend time validating that the data is correct. Otherwise, you can make erroneous decisions, especially on troubleshooting activities, because the timestamps and other information is incorrect. Taps are also easy, since they are essentially set and forget.
As a general note, if you choose to use SPAN ports, be careful when mixing data from Taps and SPAN ports. As long as you tag or segregate the data, you’ll know what set of data is exact data versus data that could be/has been altered.
In addition, here are some best practices to follow for tap placement:
- Use taps where you can to ensure that you get the best data possible as fast as possible
- Tap your network ingress and egress points
- Tap any known choke points
- Revisit your visibility architecture plan to understand where your data is coming from
If you’re looking for more information, download these whitepapers to get other tips: