Added advanced Heavy Hitter Detection example (#136)

* Added advanced Heavy Hitter Detection example

* Changed directory location

* Restored skeleton version

* Added files for common run infra with the other tutorials

* Updated readme

* Autogenerate setup rules

* Commends in simple_router.p4

* Fix typos

* Removed commended out lines

Author: gnikol

Teaching/Stanford_CS344_2018/.gitignore

@@ -0,0 +1 @@
+simple_router.config

Teaching/Stanford_CS344_2018/README.md

@@ -0,0 +1,126 @@
+# Instructions
+
+## Introduction
+
+In this tutorial, you will implement a heavy hitter detection filter.
+
+Network flows typically have a fairly wide distribution in terms of the 
+data they transmit, with most of the flows sending little data and few
+flows sending a lot. The latter flows are called heavy hitters, and they
+often have a detrimental effect to network performance. This is
+because they cause congestion, leading to significantly increased completion
+times for small, short-lived flows. Detecting heavy hitters allows us to treat them 
+differently, e.g. we can put their packets in low priority queues, allowing
+packets of other flows to face little or no congestion.
+
+In this example, you will implement a heavy hitter detection filter within 
+a router. You can find a skeleton of the program in simple_router.p4. In that
+file, you have to fill in the parts that are marked with TODO.
+
+This example is based on [count-min sketch](http://theory.stanford.edu/~tim/s15/l/l2.pdf).
+In fact, we use two count-min sketches which are reset with an offset
+equal to their half-life. With every new packet coming in, we update
+the values of both sketches but we use only the ones of the least 
+recently reset one to decide whether a packet belongs to a heavy hitter
+flow or not.
+
+> **Spoiler alert:** There is a reference solution in the `solution`
+> sub-directory. Feel free to compare your implementation to the
+> reference.
+
+
+## Step 1: Run the (incomplete) starter code
+
+The directory with this README also contains a skeleton P4 program,
+`simple_router.p4`, which implements a simple router. Your job will be to
+extend this skeleton program to properly implement a heavy hitter
+detection filter.
+
+Before that, let's compile the incomplete `simple_router.p4` and bring
+up a switch in Mininet to test its behavior.
+
+1. In your shell, run:
+   ```bash
+   ./run.sh
+   ```
+   This will:
+   * create a p4app application,
+   * compile `simple_switch.p4`,
+   * generate control plane code,
+   * start a Mininet instance with one switch (`s1`) conected to
+	 two hosts (`h1` and `h2`).
+   * install the control plane code to your switch,
+   * The hosts are assigned IPs of `10.0.0.10` and `10.0.1.10`.
+
+2. You should now see a Mininet command prompt. Run ping between
+   `h1` and `h2` to make sure that everything runs correctly:
+   ```bash
+   mininet> h1 ping h2
+   ```
+   You should see all packets going through.
+
+3. Type `exit` to leave each Mininet command line.
+
+### A note about the control plane
+
+A P4 program defines a packet-processing pipeline, but the rules
+within each table are inserted by the control plane. When a rule
+matches a packet, its action is invoked with parameters supplied by
+the control plane as part of the rule.
+
+In this exercise, we have already implemented the control plane
+logic for you. As part of invoking `run.sh`, a set of rules is generated
+by `setup.py` and when bringing up the Mininet instance, these
+packet-processing rules are installed in the tables of
+the switch. These are defined in the `simple_router.config` file.
+
+## Step 2: Implement the heavy hitter detection filter
+
+The `simple_router.p4` file contains a skeleton P4 program with key pieces of
+logic replaced by `TODO` comments. Your implementation should follow
+the structure given in this file, just replace each `TODO` with logic
+implementing the missing piece.
+
+More specifically, you need to implement the main actions used within
+the heavy hitter detection block. In this example, when our filter
+classifies a packet as belonging to a heavy hitter flow, it marks
+it as such and then the switch drops it before reaching the 
+egress control.
+
+## Step 3: Run your solution
+
+Our heavy hitter filter requires periodic reset of the registers of the
+count-min sketches. Running:
+```bash
+bash filter_reset.sh
+```
+in a terminal window does that periodic reset for you.
+
+The filter currently allows 1000 bytes/sec (you can change that value
+in `setup.py`).
+
+In another terminal window, run:
+```bash
+./run.sh
+```
+
+In the minigraph window, you can try:
+```
+h1 ping -s 80 -i 0.1 h2
+```
+With this command h1, sends a packet with a total IP length
+of 100 bytes every 100 ms. When you run this command, you
+shouldn't see any drops. If on the other hand you run:
+```
+h1 ping -s 80 -i 0.05 h2
+```
+h1 sends a packet every 50 ms, which puts the flow above
+the filter limit. In this case you will observe that about
+half of the packets send by h1 are being dropped at the switch.
+
+### Next steps
+Check out the code in `setup.py` and `filter_reset.sh`. By changing
+the constants in those, you can experiment with different 
+heavy hitter threshold levels, count-min sketch sizes and the accuracy
+of the throughput approximation.
+

Teaching/Stanford_CS344_2018/filter_reset.sh

@@ -0,0 +1,26 @@
+#!/bin/sh
+CONTAINER_ID=`docker ps | tail -n 1 | cut -d ' ' -f 1`
+ACTIVE_FILTER='A'
+
+while true; do
+    CUR_TIME=`echo "get_time_elapsed" | docker exec -i $CONTAINER_ID simple_switch_CLI | grep Runtime | head -n 1 | cut -d ':' -f 2`
+    CUR_TIME=${CUR_TIME}000
+    echo $CUR_TIME
+    echo "register_write last_reset_time 0 $CUR_TIME" | docker exec -i $CONTAINER_ID simple_switch_CLI
+    if [ $ACTIVE_FILTER == 'A' ] ; then
+        echo "register_write is_a_active 0 1"
+        echo "register_reset hashtable_b0" | docker exec -i $CONTAINER_ID simple_switch_CLI
+        echo "register_reset hashtable_b1" | docker exec -i $CONTAINER_ID simple_switch_CLI
+        echo "register_reset hashtable_b2" | docker exec -i $CONTAINER_ID simple_switch_CLI
+        echo "register_reset hashtable_b3" | docker exec -i $CONTAINER_ID simple_switch_CLI
+        ACTIVE_FILTER='B'
+    else
+        echo "register_write is_a_active 0 0"
+        echo "register_reset hashtable_a0" | docker exec -i $CONTAINER_ID simple_switch_CLI
+        echo "register_reset hashtable_a1" | docker exec -i $CONTAINER_ID simple_switch_CLI
+        echo "register_reset hashtable_a2" | docker exec -i $CONTAINER_ID simple_switch_CLI
+        echo "register_reset hashtable_a3" | docker exec -i $CONTAINER_ID simple_switch_CLI
+        ACTIVE_FILTER='A'
+    fi
+    sleep 4
+done

Teaching/Stanford_CS344_2018/header.p4

@@ -0,0 +1,83 @@
+#ifndef __HEADER_P4__
+#define __HEADER_P4__ 1
+
+struct ingress_metadata_t {
+    bit<32> nhop_ipv4;
+}
+
+header ethernet_t {
+    bit<48> dstAddr;
+    bit<48> srcAddr;
+    bit<16> etherType;
+}
+
+header ipv4_t {
+    bit<4>  version;
+    bit<4>  ihl;
+    bit<8>  diffserv;
+    bit<16> totalLen;
+    bit<16> identification;
+    bit<3>  flags;
+    bit<13> fragOffset;
+    bit<8>  ttl;
+    bit<8>  protocol;
+    bit<16> hdrChecksum;
+    bit<32> srcAddr;
+    bit<32> dstAddr;
+}
+
+header tcp_t {
+    bit<16> srcPort;
+    bit<16> dstPort;
+    bit<32> seqNo;
+    bit<32> ackNo;
+    bit<4> dataOffset;
+    bit<4> res;
+    bit<8> flags;
+    bit<16> window;
+    bit<16> checksum;
+    bit<16> urgentPtr;
+}
+
+header udp_t {
+    bit<16> srcPort;
+    bit<16> dstPort;
+    bit<16> hdrLength;
+    bit<16> checksum;
+}
+
+struct hhd_t {
+    @name("filter_age")
+    bit<48> filter_age;
+    bit<32> value_a0;
+    bit<32> value_a1;
+    bit<32> value_a2;
+    bit<32> value_a3;
+    bit<32> value_b0;
+    bit<32> value_b1;
+    bit<32> value_b2;
+    bit<32> value_b3;
+    bit<32> threshold;
+    bit<1>  is_a_active;
+    bit<1>  is_heavy_hitter;
+}
+
+struct metadata {
+    @name("ingress_metadata")
+    ingress_metadata_t   ingress_metadata;
+    @name("hhd")
+    hhd_t hhd;
+}
+
+struct headers {
+    @name("ethernet")
+    ethernet_t ethernet;
+    @name("ipv4")
+    ipv4_t     ipv4;
+	@name("tcp")
+	tcp_t tcp;
+	@name("udp")
+	udp_t udp;
+}
+
+#endif // __HEADER_P4__

Teaching/Stanford_CS344_2018/p4app.json

@@ -0,0 +1,10 @@
+{
+  "program": "simple_router.p4",
+  "language": "p4-16",
+  "targets": {
+    "mininet": {
+      "num-hosts": 2,
+      "switch-config": "simple_router.config"
+    }
+  }
+}

Teaching/Stanford_CS344_2018/parser.p4

@@ -0,0 +1,51 @@
+parser ParserImpl(packet_in packet, out headers hdr, inout metadata meta, inout standard_metadata_t standard_metadata) {
+    state start {
+        transition parse_ethernet;
+    }
+
+    state parse_ethernet {
+        packet.extract(hdr.ethernet);
+        transition select(hdr.ethernet.etherType) {
+            16w0x800: parse_ipv4;
+            default: accept;
+        }
+    }
+
+    state parse_ipv4 {
+        packet.extract(hdr.ipv4);
+        transition select(hdr.ipv4.protocol) {
+            8w0x6: parse_tcp;
+            default: accept;
+        }
+    }
+
+    state parse_tcp {
+        packet.extract(hdr.tcp);
+        transition accept;
+    }
+}
+
+control DeparserImpl(packet_out packet, in headers hdr) {
+    apply {
+        packet.emit(hdr.ethernet);
+        packet.emit(hdr.ipv4);
+        packet.emit(hdr.tcp);
+    }
+}
+
+control verifyChecksum(inout headers hdr, inout metadata meta) {
+    apply { }
+}
+
+control computeChecksum(inout headers hdr, inout metadata meta) {
+    apply {
+        update_checksum(
+                hdr.ipv4.isValid(),
+                { hdr.ipv4.version, hdr.ipv4.ihl, hdr.ipv4.diffserv,
+                hdr.ipv4.totalLen, hdr.ipv4.identification,
+                hdr.ipv4.flags, hdr.ipv4.fragOffset, hdr.ipv4.ttl,
+                hdr.ipv4.protocol, hdr.ipv4.srcAddr, hdr.ipv4.dstAddr },
+                hdr.ipv4.hdrChecksum,
+                HashAlgorithm.csum16);
+    }
+}

Teaching/Stanford_CS344_2018/run.sh

@@ -0,0 +1,6 @@
+P4APPRUNNER=../utils/p4apprunner.py
+python setup.py
+mkdir -p build
+tar -czf build/p4app.tgz * --exclude='build'
+#cd build
+sudo python $P4APPRUNNER p4app.tgz --build-dir ./build

Teaching/Stanford_CS344_2018/setup.py

@@ -0,0 +1,21 @@
+import os
+from shutil import copyfile
+
+unit_duration = 20 # log_2 of unit duration (so 2**unit_duration)
+total_time_bits = 48
+log_units = 3 # log_2 of number of units
+units = 2**log_units
+threshold = 8*1000.0 # in bytes
+
+copyfile('simple_router.config.template', 'simple_router.config')
+
+with open('simple_router.config', 'a') as fd:
+    time_mask = (2**(unit_duration+log_units)-1) - (2**unit_duration -1)
+    for unit in range(units):
+        time_value = unit*2**unit_duration
+        if unit < units/2:
+            unit_threshold = int((unit+1) * threshold / units + threshold/2 )
+        else:
+            unit_threshold = int((unit+1) * threshold / units)
+        fd.write('table_add threshold_table set_threshold %d&&&%d => %d 0\n' % (time_value, time_mask, unit_threshold))
+

Teaching/Stanford_CS344_2018/simple_router.config.template

@@ -0,0 +1,12 @@
+set_crc16_parameters calc_2 0x1021 0xffff 0x0000 false false
+set_crc32_parameters calc_0 0x4c11db7 0xffffffff 0x00000000 false false
+table_set_default send_frame egress_drop
+table_set_default forward ingress_drop
+table_set_default ipv4_lpm ingress_drop
+table_add send_frame rewrite_mac 1 => 00:aa:bb:00:00:00
+table_add send_frame rewrite_mac 2 => 00:aa:bb:00:00:01
+table_add forward set_dmac 10.0.0.10 => 00:04:00:00:00:00
+table_add forward set_dmac 10.0.1.10 => 00:04:00:00:00:01
+table_add ipv4_lpm set_nhop 10.0.0.10/32 => 10.0.0.10 1
+table_add ipv4_lpm set_nhop 10.0.1.10/32 => 10.0.1.10 2
+table_add drop_heavy_hitter heavy_hitter_drop 1 0