Lab: networking

Your Job (hard)

This is definitely a hard and time-consuming lab if you read through the device manual for programming. However, the given hints are useful enough to finish this task. Instead of repeating the hints, I will write about the code structures in the e1000 driver.

Descriptor & Buffer

The e1000 device uses direct memory access (DMA) technique to access the transmitted/received data in RAM. In order to specify the memory address of the data, driver provides the information in descriptors.

• Each descriptor is a 16-byte structure:

  

  The packet received will be placed at the buffer address. The remaining bytes are necessary information about the received packet.

  

  E1000 will read length bytes starting from the buffer address to its internal FIFO buffer and transmit the packet. Sending commands is in the cmd byte.

• The buffer address is the starting position of valid data (head) in the mbuf.buf:

  struct mbuf {
    struct mbuf  *next; // the next mbuf in the chain
    char         *head; // the current start position of the buffer
    unsigned int len;   // the length of the buffer
    char         buf[MBUF_SIZE]; // the backing store
  };
  

  The net device has no idea about mbuf, driver code manages its allocation and free. (see mbufalloc() and mbuffree(), which calls the physical memory manager in kalloc.c)

Descriptor Ring

To maximize throughput, e1000 uses a circular array of descriptors (or buffers) to support consecutive operations. Device can access its base address, head/tail pointers and length from memory-mapped registers.

	// [E1000 14.5] Transmit initialization
  memset(tx_ring, 0, sizeof(tx_ring));
  for (i = 0; i < TX_RING_SIZE; i++) {
    tx_ring[i].status = E1000_TXD_STAT_DD;
    tx_mbufs[i] = 0;
  }
  regs[E1000_TDBAL] = (uint64) tx_ring;
  regs[E1000_TDLEN] = sizeof(tx_ring);
  regs[E1000_TDH] = regs[E1000_TDT] = 0;
  
  // [E1000 14.4] Receive initialization
  memset(rx_ring, 0, sizeof(rx_ring));
  for (i = 0; i < RX_RING_SIZE; i++) {
    rx_mbufs[i] = mbufalloc(0);
    if (!rx_mbufs[i])
      panic("e1000");
    rx_ring[i].addr = (uint64) rx_mbufs[i]->head;
  }
  regs[E1000_RDBAL] = (uint64) rx_ring;
  regs[E1000_RDH] = 0;
  regs[E1000_RDT] = RX_RING_SIZE - 1;
  regs[E1000_RDLEN] = sizeof(rx_ring);

It is a producer-consumer model:

• head pointer is managed by hardware directly, specifying which buffer is writing or reading by the device. tail pointer should be updated by driver software. It points to the buffer that has just already been processed(read or written)
• Hardware will check the tail and head pointer to see whether the buffer array is full (receive) or empty (transmit). If not, hardware will finish his job, mark this descriptor in the E1000_TXD_STAT_DD bit and increment the head pointer.
• Software must check whether the buffer array is full (transmit) or empty (receive) before operating, which is implemented by checking the E1000_TXD_STAT_DD bit for each descriptor. Software should increment the tail pointer after processing.

Buffer Pointers

For each descriptor, there is a corresponding buffer pointer mbuf *. It is the bridge between net code stack and hardware.

Note: The buffer pointers are static, but the actual mbuf data is dynamic!

static struct tx_desc tx_ring[TX_RING_SIZE] __attribute__((aligned(16)));
static struct mbuf *tx_mbufs[TX_RING_SIZE];

static struct rx_desc rx_ring[RX_RING_SIZE] __attribute__((aligned(16)));
static struct mbuf *rx_mbufs[RX_RING_SIZE];

Transmit a packet: