BWCTL Architecture

$Date: 2009-01-26 08:59:22 -0500 (Mon, 26 Jan 2009) $

Components Overview

In the general case, BWCTL works by having a bwctld process on each of the endpoint test systems. The bwctl client contacts each of those daemons and requests a test. That request includes an indication of the desired time period as well as a full parameterization of the desired Iperf, Nuttcp or Thrulay test. bwctld is responsible for implementing the policy and scheduling constraints.

General Case

bwctld has been developed as a classic accept/fork daemon. The master daemon process listens for new network connections and also manages the resources for all child bwctld processes. Once a connection comes in, bwctld forks a child process to handle that request. The child process deals with all encryption and communication issues with the client, as well as dealing with all static resource limits. Static resource limits are those not dependent upon what is currently happening on the node. For example, the request broker can easily determine if the given client is allowed to do UDP tests without talking to the master daemon. Once the request broker process determines the request is valid, it makes a request to the master daemon for the resources and time period requested by the client. If the master daemon has the resources available (including the open time period), it grants the request.

Scheduling Requests

The time period for the test is always requested by the client. It makes a request to the request broker process on each endpoint of the test in turn. The time period is requested by specifying two timestamps. The earliest time and the latest time the client is willing to settle for. The request broker either returns a tentative reservation with the first open time that will fulfill the parameters of the test or a request denied message. The client can then use the tentative reservation time as the earliest time in the request to the request broker on the other host. Eventually, an agreed upon time will be reached or the latest time will be reached. If the request broker can not fulfill the request before the latest time specified in the request, it will return a server too busy message. If the client gets the same time from both servers, it must then confirm the reservation with a start session message. The start session message must be received before a configurable timeout period and before the reservation time, or the server will disallow the reservation.

Scheduling Algorithm

The bwctld parent resource broker process holds the master schedule. When requests come in from clients, the request broker process does the non-global authorization checks and then passes the global authorization checks up to the resource broker including the request for a given time slot. The scheduling itself is simply a first-fit algorithm with limits imposed on how far it is willing to schedule into the future.

Each time slot that is requested is padded to allow for time synchronization issues. The algorithm for the padding is dynamic and depends upon the following factors:

The algorithm is:
Where A is a small constant. (Currently, one second in the code.)

This amount of time is padded before and after each time slot that is requested. The algorithm was designed to continue to work in the case when the test peer is not as well synchronized and is perhaps halfway around the globe.

The padding is placed before and after the test slot because there is 2 RTT of communication before and after each test. For well synchronized hosts that are fairly near one another, this algorithm usually provides 2-3 seconds of padding between each test (time slot). This could potentially be reduced by making the A constant smaller. That constant represents the amount of time it takes a host to context switch, start processes, and includes how quickly the throughput tester notices that its time is up. Experimentation has shown that even with 1 second of padding, the throughput testers sometimes need to be killed off.

Test Execution

Once the client sends the the start session message, the request broker forks off a peer agent that is responsible for verifying the time offset to the other endpoint of the test and initializing the communication socket that will be used to trade results of the test.

If the test endpoint systems have a reasonably close idea of the time, and they can communicate, the peer agent forks off the test process. The test process waits until the scheduled start time and then executes the desired throughput tester with the correct command line parameters. Note: Experience is showing that a closer integration with the testing process (Iperf, Nuttcp or Thrulay) may be needed in the future.

End User Workstation Configuration

In the event that the local host is one of the endpoints and there is no local bwctld running, bwctl will spawn additional processes to execute the local side of the test directly. In this case, a diagram of the processes involved would look like the following:

Local bwctld not available

The same basic communication takes place between the different processes in this alignment. The main difference is that the client has no need of a resource broker. This is the expected usage scenario for end user workstations to run tests to well defined BWCTL measurement beacons.

* The images above shows a dashed blue line around the processes that occur on each host.



$Id: architecture.html 525 2009-01-26 13:59:22Z aaron $