Flow

Handler type: flow

• Example of policy
• Metrics Group
• Filters
• Configurations

Example of policy with input flow and handler FLOW

handlers:
    window_config:
        deep_sample_rate: 95
        num_periods: 6
    modules:
        my_flow:
            type: flow
            config:
                sample_rate_scaling: false
                topn_count: 7
                device_map:
                  192.168.3.32:
                    name: Device1
                    description: This is a device map example
                    interfaces:
                      2:
                        name: Prov1
                        description: This is an interface map example
                summarize_ips_by_asn: true
                exclude_unknown_asns_from_summarization: true
                exclude_asns_from_summarization:
                  - 16509
                exclude_ips_from_summarization:
                  - 192.168.3.32/32
            metric_groups:
                enable:
                    - cardinality
                    - counters
                    - top_geo
                    - by_packets
                    - by_bytes
                    - conversations
                    - top_ports
                    - top_ips
                    - top_interfaces
                    - top_ips_ports
                    - top_tos
            filter:
                only_ports:
                    - 10853
                    - 10860-10890
                only_device_interfaces:
                    192.0.2.10:
                        - 2
                only_directions: ["in"]
                only_ips:
                  - 192.168.2.1/24
                  - 192.0.2.38/32
                geoloc_notfound: true
                asn_notfound: true
input:
  input_type: flow
  tap: default_flow
kind: collection

Summarizing by ASN

A busy flow collector produces far more distinct IPs than are useful to store. Summarizing by ASN collapses them, and the exclusions keep the aggregate meaningful: drop the ASNs you do not want folded in, and drop the addresses that resolve to no ASN at all rather than letting them pile into one bucket.

only_device_interfaces narrows this further to named devices and their SNMP interface indices. It is a map keyed by device IP.

handlers:
  modules:
    flow_by_asn:
      type: flow
      config:
        summarize_ips_by_asn: true
        exclude_unknown_asns_from_summarization: true
        exclude_asns_from_summarization:
          - 64496
        subnets_for_summarization:
          - 0.0.0.0/24
          - ::/64
      filter:
        only_device_interfaces:
          192.0.2.1: [1, 2, 3]
          192.0.2.2: [1, 2, 3]
      metric_groups:
        disable:
          - all
        enable:
          - counters
          - by_bytes
          - top_ports
          - top_ips
          - top_tos
input:
  input_type: flow
  tap: my_flow_tap
kind: collection

Metrics Group

• Check the flow metrics belonging to each group

Filters

only_device_interfaces

Type: map

Input: FLOW

only_device_interfaces filters data by only retaining flows coming from the specific devices and interfaces defined in this filter.

The difference between only_device_interfaces and only_ips is that only_ips filters based on the IPs observed inside the flows, while only_device_interfaces filters based on the device and interface sending the flows.

The only_device_interfaces filter usage syntax is:

only_device_interfaces:
  device:
    - interface

Example:

only_device_interfaces:
  192.0.2.10:
    - 2 #interface index as int
    - 4-10 #a range: interfaces 4 to 10 inclusive
    - "1" #interface index as str
  192.0.2.38: [9, 4-10]
  192.0.2.32:
    - "*" #all interfaces

only_directions

Type: str[]

only_directions filters data by its direction. Options are: "in" and/or "out".

The only_directions filter usage syntax is:

only_directions: [str]

Example:

only_directions:
    - "in"

only_ips

Type: str[]

Input: FLOW

To filter data only from certain source OR destination, you can use only_ips filter, for which CIDR (Inter-Domain Routing Classes) ranges are supported.

The only_ips filter usage syntax is:

only_ips:
  - array

Example:

only_ips:
  - 192.168.1.1/24
  - 192.0.2.38/32

only_ports

Type: str[]

Input: FLOW

only_ports filter only filters data being sent to or received on one of the selected TCP/UDP ports (or range of ports).

The only_ports filter usage syntax is:

only_ports:
  - array

Example:

only_ports:
  - 10853 #port can be passed as int
  - "10854" #port can be passed as str
  - 10860-10890 #range from 10860 to 10890. All ports in this interval will be accepted

geoloc_notfound

Type: bool

Input: FLOW

The source and destination IPs are used to determine the geolocation to know where the data is from and where it is going. When the IPs refer to a region found in the standard databases, the city, state and country (approximated) are returned. However, when it is not possible to determine the IP geolocation, a not found is returned.

The geoloc_notfound filter usage syntax is:

geoloc_notfound: true

asn_notfound

Type: bool

Input: FLOW

Based on source and destination IP, it is possible to determine the ASN (Autonomous System Number). When the IP of the source or destination belongs to some not known ASN in the standard databases, a not found is returned.

The asn_notfound filter usage syntax is:

asn_notfound: true

Configurations

• sample_rate_scaling: bool

• enrichment: bool

• device_map: map

• summarize_ips_by_asn: bool

• exclude_asns_from_summarization: str[]

• exclude_unknown_asns_from_summarization: bool

• subnets_for_summarization: str[]

• exclude_ips_from_summarization str[]

• recorded_stream. Presence-based.

• Abstract configurations.

sample_rate_scaling

By default, flow metrics are generated by an approximation based on sampling the data. 1 packet every N is analyzed and the prediction of the entire population is made from the sample. If you want to see exactly all the exact data, you can disable sample_rate_scaling.

The sample_rate_scaling filter usage syntax is:

sample_rate_scaling: false

enrichment

Enrichment applies the device_map settings to the flow data.

It is on by default: it is active unless the key is set to false, so setting it to true changes nothing. To turn it off:

enrichment: false

device_map

This configuration allows the user to assign a custom name to devices and interfaces, and it applies only while enrichment is on, which it is unless enrichment: false is set.

The device_map configuration usage syntax is:

device_map:
  device_ip:
    name: "str" #name is required
    description: "Optionally set a description"
    interfaces: #required
        interface:
            name: "str" #required
            description: "Optionally set a description"

Example:

device_map:
  192.168.2.32:
    name: Cisco
    description: This is a device map example
    interfaces:
      2:
        name: GoogleProv
        description: This is an interface map example

Summarization is a useful strategy for visualization, but also for decreasing the cardinality of the data, and two types of summarization are supported: by asn and by subnets, and summarization by ASN is dominant over subnet, i.e. If both configurations are present, only the IPs of an excluded asn or an unknown asn (if exclude_unknown_asns_from_summarization is true) will be summarized by subnet.

summarize_ips_by_asn

When True, it summarizes data by ASN (Autonomous System Number).

The summarize_ips_by_asn configuration usage syntax is:

summarize_ips_by_asn: true

exclude_asns_from_summarization

This configuration must be used in conjunction with summarize_ips_by_asn, in order to exclude ASNs from summarization. In this case, packets transacted by excluded ASNs will be exposed by IPs.

The exclude_asns_from_summarization configuration usage syntax is:

exclude_asns_from_summarization:
  - 8075
  - 16509

exclude_unknown_asns_from_summarization

This configuration must be used in conjunction with summarize_ips_by_asn, in order to expose IPs from packets transacted by unknown ASNs.

The exclude_unknown_asns_from_summarization configuration usage syntax is:

exclude_unknown_asns_from_summarization: true

subnets_for_summarization

This configuration allows the summarization of flow data by subnets. Attention: This configuration will only work properly if the summarize_ips_by_asn configuration is not set for the IP, since summarize_ips_by_asn is dominant.

The subnets_for_summarization configuration usage syntax is:

subnets_for_summarization:
  - 192.168.2.1/24

Tip:

It is possible to define a summary pattern by defining the CIDR only. In this way, all present IPs will be summarized. For this to be done, just pass the default host mask for IPv4 or/and IPv6 and the desired CIDR for grouping. This pattern has less priority than the explicitly defined subnets, so only IPs not belonging to any set subnet will be summarized following the general pattern.

subnets_for_summarization:
  - 0.0.0.0/16
  - ::/64

exclude_ips_from_summarization

This configuration must be used in conjunction with summarize_ips_by_asn or subnets_for_summarization and will remove the specified IPs from the summarization.

The exclude_ips_from_summarization configuration usage syntax is:

exclude_ips_from_summarization:
  - 192.168.2.1/31

recorded_stream

The key is presence-based: the handler checks only whether recorded_stream is set, never its value, so recorded_stream: false still enables it. Omit the key entirely to disable.

This configuration is useful when a pcap_file is used in taps/input configuration. Set it to True when you want to load an offline traffic (from a pcap_file).

The recorded_stream configuration usage syntax is:

recorded_stream: true