ALTQ API draft: (version 04) September 27 1999 last update: $Date: 2006/09/28 03:00:40 $ 1. QOP (queue operation) API overview The QOP API is an application programming interface of ALTQ queueing disciplines and provides a uniform interface to different queueing disciplines. The API is designed to provide: - uniform handling of different disciplines - a single configuration file for different disciplines on multiple interfaces - memory management - error handling The QOP API consists of a common module and a set of discipline specific modules. Many queue operaions are discipline independent (e.g., enabling ALTQ) and thus done through the common module. The common module also manages: reading config file / command inputs memory allocation / deallocation interface/class/filter name mapping Whereas, some operations are intrinsically discipline specific since they need discipline specific parameters. Currently, the following discipline specific operations are defined: - attach a discipline to an interface - create a class (when a discipline has classes) - modify an existing class Additional sets of APIs are built on top of the QOP API: QCMD (queue command) API: The qcmd api is a string-based API that is designed to be called from the interpreter. (the interpreter reads the configuration file, or command inputs for the interactive mode.) The qcmd api is much simpler than the qop api so that simple applications will use the qcmd api instead of the qop api. rsvpd TCif: The rsvpd implementation from ISI has a TCif (traffic control interface). The module translates the rsvp parameters to the corresponding discipline specific parameters. diffserv PEP TCif: we expect a future diffserv PEP (policy enforcement point) implementation will need a common TCif similar to rsvpd. common module queueing discipline dependent modules +-------------+ | command | | interpreter | +-------------+ | | | | qcmd api +=============+ +=============+ rsvpd TCif +============+ | qcmd -> qop | | qcmd -> qop | (or diffserv|rsvp -> hfsc| +-------------+ +-------------+ PEP TCif) +------------+ | | qop api +=============+ +=============+ | | | | | |<--- | | | | | | | | +-------------+ | |-----------+ +-------------+ | | +=============+ qdisc system if | queue disc | | dependent | | syscalls | +-------------+ Figure 1: API model 2. QCMD functions Naming convention: common: qcmd_functionname(); (e.g., qcmd_enable()) qdisc specific: qcmd_qdiscname_functionname(); (e.g., qcmd_cbq_add_class()) 2.1 Common QCMD functions Interface Operations int qcmd_enable(const char *ifname); int qcmd_disable(const char *ifname); int qcmd_delete_if(const char *ifname); int qcmd_clear_hierarchy(const char *ifname); "qcmd_enable" and "qcmd_disable" enables or disables ALTQ on the specified interface. "qcmd_delete_if" deletes the interface; all the classes and filters associated with this interface are destroyed. "qcmd_clear_hierarchy" brings the interface back to the initial state (the state just after qcmd_xxx_add_if()). int qcmd_enableall(void); int qcmd_disableall(void); "qcmd_enableall" and "qcmd_disableall" enables or disables ALTQ on all the configured interface. int qcmd_config(void); "qcmd_config" reads the config file and executes commands in the config file. int qcmd_init(void); "qcmd_init" reads the config file and sets up interfaces, classes and filters, and then, enables all the interfaces. "qcmd_init" is a shorthand of qcmd_config(); qcmd_enableall(); int qcmd_destroyall(void); "qcmd_destroyall" disables ALTQ and frees all the memory allocated. int qcmd_restart(void); "qcmd_restart" is a shorthand of qcmd_destroyall(); qcmd_init(); and usually called when SIGHUP is received. Class Operations int qcmd_delete_class(const char *ifname, const char *class_name); Filter Operations int qcmd_add_filter(const char *ifname, const char *class_name, const char *filter_name, const struct flow_filter *filter); int qcmd_delete_filter(const char *ifname, const char *clname, const char *flname); 2.2 CBQ QCMD functions int qcmd_cbq_add_if(const char *ifname, u_int bandwidth, int is_wrr, int efficient); int qcmd_cbq_add_class(const char *ifname, const char *class_name, const char *parent_name, const char *borrow_name, u_int pri, u_int bandwidth, u_int maxdelay, u_int maxburst, u_int minburst, u_int av_pkt_size, u_int max_pkt_size, int admission_type, int flags); int qcmd_cbq_modify_class(const char *ifname, const char *class_name, u_int pri, u_int bandwidth, u_int maxdelay, u_int maxburst, u_int minburst, u_int av_pkt_size, u_int max_pkt_size, int flags); 2.3 H-FSC QCMD functions int qcmd_hfsc_add_if(const char *ifname, u_int bandwidth, int flags); int qcmd_hfsc_add_class(const char *ifname, const char *class_name, const char *parent_name, u_int m1, u_int d, u_int m2, int qlimit, int flags); int qcmd_hfsc_modify_class(const char *ifname, const char *class_name, u_int m1, u_int d, u_int m2, int sctype); 3. QOP structures The QOP API defines 3 structures: one for interfaces, one for classes, and one for filters. The interface structure and the class structure have a pointer to a discipline specific data. 3.1 Common QOP structures #include #include "altq_qop.h" /* * interface info */ struct ifinfo { LIST_ENTRY(ifinfo) next; /* next entry on iflist */ char *ifname; /* interface name */ u_int bandwidth; /* bandwidth in bps */ u_int ifmtu; /* mtu of the interface */ u_int ifindex; /* interface index */ int enabled; /* hfsc on/off state */ LIST_HEAD(, classinfo) cllist; /* class list */ LIST_HEAD(, fltrinfo) fltr_rules; /* filter rule list */ struct classinfo *resv_class; /* special class for rsvp */ /* discipline info */ struct qdisc_ops *qdisc; /* qdisc system interface */ void *private; /* discipline specific data */ int (*enable_hook)(struct ifinfo *); int (*delete_hook)(struct ifinfo *); }; /* * class info */ struct classinfo { LIST_ENTRY(classinfo) next; /* next entry on cllist of ifinfo */ u_long handle; /* class handle */ char *clname; /* class name */ struct ifinfo *ifinfo; /* back pointer to ifinfo */ struct classinfo *parent; /* parent class */ struct classinfo *sibling; /* sibling class */ struct classinfo *child; /* child class */ LIST_HEAD(, fltrinfo) fltrlist; /* filters for this class */ void *private; /* discipline specific data */ int (*delete_hook)(struct classinfo *); }; /* * filter info */ struct fltrinfo { LIST_ENTRY(fltrinfo) next; /* next entry on fltrlist of classinfo */ LIST_ENTRY(fltrinfo) nextrule; /* next entry on fltr_rules of ifinfo */ u_long handle; /* filter handle */ char *flname; /* filter name, if specified */ struct flow_filter fltr; /* filter value */ struct classinfo *clinfo; /* back pointer to classinfo */ /* for consistency check */ int line_no; /* config file line number */ int dontwarn; /* supress warning msg */ }; 3.2 CBQ QOP structure #include "qop_cbq.h" /* * cbq private ifinfo structure */ struct cbq_ifinfo { struct classinfo *root_class; /* root class */ struct classinfo *default_class; /* default class */ struct classinfo *ctl_class; /* control class */ double nsPerByte; /* bandwidth in ns per sec */ int is_wrr; /* use weighted-round robin */ int is_efficient; /* use work-conserving */ }; /* * cbq private classinfo structure */ struct cbq_classinfo { u_int bandwidth; /* bandwidth in bps */ u_int allocated; /* bandwidth used by children */ u_int maxdelay; u_int maxburst; u_int minburst; u_int av_pkt_size; u_int max_pkt_size; cbq_class_spec_t class_spec; /* class parameters */ }; 3.2 H-FSC QOP structure #include "qop_hfsc.h" /* * hfsc private ifinfo structure */ struct hfsc_ifinfo { struct classinfo *root_class; /* root class */ struct classinfo *default_class; /* default class */ }; /* * hfsc private classinfo structure */ struct hfsc_classinfo { struct service_curve rsc; /* real-time service curve */ struct service_curve fsc; /* fair service curve */ gsc_head_t gen_rsc; /* generalized real-time sc */ gsc_head_t gen_fsc; /* generalized fsc */ int qlimit; int flags; }; 4. QOP functions Naming convention: common: qop_functionname(); (e.g., qop_enable()) qdisc specific: qop_qdiscname_functionname(); (e.g., qop_cbq_add_class()) 4.1 Common QOP functions Interface Operations int qop_enable(struct ifinfo *ifinfo); int qop_disable(struct ifinfo *ifinfo); int qop_delete_if(struct ifinfo *ifinfo); int qop_clear_hierarchy(struct ifinfo *ifinfo); "qop_enable" enables ALTQ on the specified interface. "qop_disable" disables ALTQ on the specified interface. "qop_delete_if" deletes the interface; all the classes and filters associated with this interface are destroyed. "qop_clear_hierarchy" brings the interface back to the initial state (the state just after qop_xxx_add_if()). Class Operations int qop_delete_class(struct classinfo *clinfo); "qop_delete_class" deletes the specified class. Filter Operations int qop_add_filter(struct fltrinfo **fltrinfo, struct classinfo *clinfo, const char *filter_name, const struct flow_filter *filter, struct fltrinfo **conflict); int qop_delete_filter(struct fltrinfo *fltr); "qop_add_filter" adds a classifier filter to the speficied class. "filter_name" can be NULL when the filter will not be referenced later by name. if "fltrinfo" is not NULL, a pointer to the new entry is stored. [filter conflicts] When the specified filter conflicts with a previously-defined filter, "qop_add_filter" returns QOPERR_FILTER_SHADOW. When argument "conflict" is not NULL, a pointer to the conflicting filter entry is stored. 4.1 CBQ QOP functions int qop_cbq_add_if(struct ifinfo **rp, const char *ifname, u_int bandwidth, int is_wrr, int efficient); "qop_cbq_add_if" configures CBQ on the specified interface. when "is_wrr" is non zero, weighted-round robin scheduling is used. otherwise, packet-by-packet round robin is used. "efficient" enables CBQ's link efficiency mode. This means that the scheduler will send a packet from the first overlimit class it encounters of all classes of the link-sharing structure are overlimit. This will also cause the scheduler to use greater than it's assigned bandwidth, if the link is capable of more than the assigned bandwidth. int qop_cbq_add_class(struct classinfo **rp, const char *class_name, struct ifinfo *ifinfo, struct classinfo *parent, struct classinfo *borrow, u_int pri, u_int bandwidth, u_int maxdelay, u_int maxburst, u_int minburst, u_int av_pkt_size, u_int max_pkt_size, int admission_type, int flags); "qop_cbq_add_class" creates a CBQ class. "class_name" can be NULL when the class will not be referenced later by name. if "rp" is not NULL, a pointer to the new entry is stored. int qop_cbq_modify_class(struct classinfo *clinfo, u_int pri, u_int bandwidth, u_int maxdelay, u_int maxburst, u_int minburst, u_int av_pkt_size, u_int max_pkt_size, int flags); "qop_cbq_modify_class" modifies class parameters. Filters already assigned to the class are not affected. 4.2 H-FSC QOP functions int qop_hfsc_add_if(struct ifinfo **rp, const char *ifname, u_int bandwidth, int flags); "qop_hfsc_add_if" configures H-FSC on the specified interface. "bandwidth" specifies the bandwidth of the interface in bits per second. (flags parameter is not used and should be set to 0.) if "rp" is not NULL, a pointer to the new entry is stored. A special class "root" is automatically created with a linear service curve (m1 = m2 = bandwidth, d = 0). /* hfsc class flags */ #define HFCF_RED 0x0001 /* use RED */ #define HFCF_ECN 0x0002 /* use RED/ECN */ #define HFCF_RIO 0x0004 /* use RIO */ #define HFCF_DEFAULTCLASS 0x1000 /* default class */ int qop_hfsc_add_class(struct classinfo **rp, const char *class_name, struct ifinfo *ifinfo, struct classinfo *parent, struct service_curve *sc, int qlimit, int flags); "qop_hfsc_add_class" creates a H-FSC class. "class_name" can be NULL when the class will not be referenced later by name. if "rp" is not NULL, a pointer to the new entry is stored. When HFCF_RED or HFCF_ECN is specified, RED or ECN is enabled respectively. When HFCF_DEFAULTCLASS is specified, this class is set as a default class of the interface, and the classifier maps unmatched packets to this class. There must be one default class on one interface. /* service curve types */ #define HFSC_REALTIMESC 1 #define HFSC_LINKSHARINGSC 2 #define HFSC_DEFAULTSC (HFSC_REALTIMESC|HFSC_LINKSHARINGSC) int qop_hfsc_modify_class(struct classinfo *clinfo, struct service_curve *sc, int sctype); "qop_hfsc_modify_class" modifies one or both of the service curves of a class. Filters already assigned to the class are not affected. The current H-FSC maintains 2 service curves; one for real-time criteria and the other for link-sharing criteria. (The original H-FSC has only one service curve used for both criteria.) When a class is created, both service curves are initialized using the same (backward-compatible) parameters. 4.3 Misc QOP functions int is_q_enabled(const char *ifname); int is_q_acc_enabled(const char *ifname); struct ifinfo *ifname2ifinfo(const char *ifname); struct classinfo *clname2clinfo(const struct ifinfo *ifinfo, const char *clname); struct fltrinfo *flname2flinfo(const struct classinfo *clinfo, const char *flname); void print_filter(const struct flow_filter *filt); const char *qoperror(int qoperrno); 4.4 Error code All functions returns 0 if successful, or an error number if failed. /* misc system errors */ #define QOPERR_OK 0 /* no error */ #define QOPERR_SYSCALL 1 /* syscall err; see errno */ #define QOPERR_NOMEM 2 /* not enough memory */ #define QOPERR_INVAL 3 /* invalid parameter */ #define QOPERR_RANGE 4 /* out of range */ #define QOPERR_BADIF 5 /* bad interface name */ #define QOPERR_BADCLASS 6 /* bad class name */ #define QOPERR_BADFILTER 7 /* bad filter name */ /* class errors */ #define QOPERR_CLASS 8 /* class failure */ #define QOPERR_CLASS_INVAL 9 /* bad class value */ #define QOPERR_CLASS_PERM 10 /* class operation not permitted */ /* filter errors */ #define QOPERR_FILTER 11 /* filter failure */ #define QOPERR_FILTER_INVAL 12 /* bad filter value */ #define QOPERR_FILTER_SHADOW 13 /* shadows an existing filter */ /* addmission errors */ #define QOPERR_ADMISSION 14 /* admission control failure */ #define QOPERR_ADMISSION_NOBW 15 /* insufficient bandwidth */ #define QOPERR_ADMISSION_DELAY 16 /* cannot meet delay bound req */ #define QOPERR_ADMISSION_NOSVC 17 /* no service available */ /* policy errors */ #define QOPERR_POLICY 18 /* policy control failure */ #define QOPERR_MAX 18 5. Input Traffic Conditioning ALTQ supports diffserv traffic conditioning at input interfaces. A unit of traffic conditioning is called "traffic conditioning element". Traffic conditioning elements include a marker, a dropper and 2 types of meters. (currently, a shaper is not supported.) The following figure shows an example of traffic conditioning. An input packet is first passed to a classifier. The classfier maps the packet to a traffic conditioning element, a meter in this example. Then, the meter decides if the packet is in profile or not, and pass the packet to the marker or the dropper according to the result. +-----+ +-----+ | |--------> dropper | |------>| |(out profile) -------->| | | |--------> marker | | +-----+(in profile) +-----+ meter classifier 5.1 Traffic Conditioning Action A traffic conditioning element takes one of the following actions. (1) pass: no action is taken for the packet (2) drop: discard the packet (3) mark set a preconfigured value to the diffserve codepoint in the IP header (4) next: go to the next element and take its action this allows to nest traffic conditioning actions struct tc_action { int tca_code; /* e.g., TCACODE_PASS */ /* tca_code dependent variable */ union { u_long un_value; /* template */ u_int8_t un_dscp; /* diffserv code point */ u_long un_handle; /* tc action handle */ struct cdnr_block *un_next; /* next tc element block */ } tca_un; }; #define tca_value tca_un.un_value #define tca_dscp tca_un.un_dscp #define tca_handle tca_un.un_handle #define tca_next tca_un.un_next #define TCACODE_NONE 0 /* action is not set */ #define TCACODE_PASS 1 /* pass this packet */ #define TCACODE_DROP 2 /* discard this packet */ #define TCACODE_RETURN 3 /* do not process this packet */ #define TCACODE_MARK 4 /* mark dscp */ #define TCACODE_HANDLE 5 /* take action specified by handle */ #define TCACODE_NEXT 6 /* take action in the next tc element */ #define TCACODE_MAX 6 5.2 Traffic Conditioner QCMD int qcmd_cdnr_add_element(struct tc_action *rp, const char *ifname, const char *cdnr_name, struct tc_action *action); "qcmd_cdnr_add_element" creates a simple conditioner element. if rp is not NULL, the resulted action is returned. EXAMPLE (a simple marker) struct tc_action action, result; action.tca_code = TCACODE_MARK; action.tca_dscp = DSCP_EF; error = qcmd_cdnr_add_element(&result, "fxp0", "my_marker", &action); /* you can add a classifier to this conditioner */ error = qcmd_add_filter("fxp0", "my_marker", "my_filter", &filt); int qcmd_cdnr_add_tbmeter(struct tc_action *rp, const char *ifname, const char *cdnr_name, struct tb_profile *profile, struct tc_action *in_action, struct tc_action *out_action); "qcmd_cdnr_add_tbmeter" creates a token bukcet meter. if rp is not NULL, the resulted action is returned. "struct tb_profile" is used to represent the profile of a token bucket. /* simple token backet meter profile */ struct tb_profile { u_int rate; /* rate in bit-per-sec */ u_int depth; /* depth in bytes */ }; EXAMPLE (a token bukcet meter) struct tb_profile profile; struct tc_action in_action, out_action, result; profile.rate = 1000000; /* 1Mbps */ profile.depth = 32 * 1024; /* 32KB */ /* mark EF when in profile, otherwise drop */ in_action.tca_code = TCACODE_MARK; in_action.tca_dscp = DSCP_EF; out_action.tca_code = TCACODE_DROP; error = qcmd_cdnr_add_tbmeter(&result, "fxp0", "my_meter", &profile, &in_action, &out_action); int qcmd_cdnr_add_trtcm(struct tc_action *rp, const char *ifname, const char *cdnr_name, struct tb_profile *cmtd_profile, struct tb_profile *peak_profile, struct tc_action *green_action, struct tc_action *yellow_action, struct tc_action *red_action, int coloraware); "qcmd_cdnr_add_trtcm" creates a 2-rate three color marker. if rp is not NULL, the resulted action is returned. EXAMPLE (a trtcm for AF1x) struct tb_profile cmtd_profile, peak_profile; struct tc_action green_action, yellow_action, red_action, result; /* committed profile and peak profile */ cmtd_profile.rate = 1000000; /* 1Mbps */ cmtd_profile.depth = 32 * 1024; /* 32KB */ peak_profile.rate = 3000000; /* 3Mbps */ peak_profile.depth = 64 * 1024; /* 64KB */ green_action.tca_code = TCACODE_MARK; green_action.tca_dscp = DSCP_AF11; yellow_action.tca_code = TCACODE_MARK; yellow_action.tca_dscp = DSCP_AF12; red_action.tca_code = TCACODE_MARK; red_action.tca_dscp = DSCP_AF13; error = qcmd_cdnr_add_trtcm(&result, "fxp0", "my_trtcm", &cmtd_profile, &peak_profile, &green_action, &yellow_action, &red_action, 0); 5.3 Traffic Conditioner QOP int qop_cdnr_add_element(struct classinfo **rp, const char *cdnr_name, struct ifinfo *ifinfo, struct tc_action *action); "qop_cdnr_add_element" creates a simple conditioner element. if rp is not NULL, the resulted classinfo is returned. int qop_cdnr_add_tbmeter(struct classinfo **rp, const char *cdnr_name, struct ifinfo *ifinfo, struct tb_profile *profile, struct tc_action *in_action, struct tc_action *out_action); "qop_cdnr_add_tbmeter" creates a token bucket meter. if rp is not NULL, the resulted classinfo is returned. int qop_cdnr_add_trtcm(struct classinfo **rp, const char *cdnr_name, struct ifinfo *ifinfo, struct tb_profile *cmtd_profile, struct tb_profile *peak_profile, struct tc_action *green_action, struct tc_action *yellow_action, struct tc_action *red_action, int colorware); "qop_cdnr_add_tbmeter" creates a 2-rate three color marker. if rp is not NULL, the resulted classinfo is returned. int qop_cdnr_modify_tbmeter(struct classinfo *clinfo, struct tb_profile *profile); "qop_cdnr_modify_tbmeter" modifies the profile of a token bucket meter. int qop_cdnr_modify_trtcm(struct classinfo *clinfo, struct tb_profile *cmtd_profile, struct tb_profile *peak_profile, int coloraware); "qop_cdnr_modify_trtcm" modifies the profiles of a 2-rate three color marker.