USB Part 14: Custom USB Class Drivers

Designing the Custom Protocol

Before writing a single descriptor, you must design the protocol your vendor class will implement. A poorly designed protocol is impossible to extend, debug, and version. A well-designed protocol survives firmware updates, OS changes, and protocol evolution without breaking existing host applications.

Endpoint Plan

A typical vendor class interface uses three endpoint types:

• Bulk OUT (host → device): Command and data from the host to the device. The device reads from this endpoint to receive commands, configuration data, or bulk payload.
• Bulk IN (device → host): Response and data from the device to the host. The device writes to this endpoint to send command responses, measurement results, or streaming data.
• Interrupt IN (optional): Asynchronous event notification. When the device needs to notify the host of an event (alarm, status change, threshold crossing) without the host polling, an interrupt IN endpoint is ideal. Maximum packet size is 64 bytes at Full Speed.

Control Requests for Configuration

Vendor control requests are the preferred mechanism for device configuration — things like setting operating modes, reading calibration data, or querying capabilities. They use EP0 (the control endpoint, always present) and do not need a dedicated endpoint. The bRequest field is your opcode space (0x01 to 0xFF for vendor use — avoid 0x00 which is GET_STATUS):

/* Vendor bRequest opcode definitions — custom protocol v1.0 */
#define VENDOR_REQ_GET_PROTOCOL_VERSION  0x01  /* IN:  wLength=4, returns uint32_t version */
#define VENDOR_REQ_GET_CAPABILITIES      0x02  /* IN:  wLength=8, returns capability bitmask */
#define VENDOR_REQ_SET_SAMPLE_RATE       0x10  /* OUT: wValue=rate_hz (0=stop), wLength=0 */
#define VENDOR_REQ_SET_CHANNEL_MASK      0x11  /* OUT: wValue=bitmask of enabled channels */
#define VENDOR_REQ_GET_STATUS            0x20  /* IN:  wLength=4, returns status flags */
#define VENDOR_REQ_RESET_DEVICE          0x30  /* OUT: wValue=0, wLength=0, soft reset */
#define VENDOR_REQ_WRITE_EEPROM          0x40  /* OUT: wValue=addr, wIndex=len, data in wLength bytes */
#define VENDOR_REQ_READ_EEPROM           0x41  /* IN:  wValue=addr, wIndex=len, data returned */

Protocol Version Header

Every bulk packet should carry a protocol header. This is not overhead — it is what makes your protocol debuggable and evolvable:

/* Bulk packet header — prepended to every bulk OUT command and bulk IN response */
typedef struct __attribute__((packed)) {
    uint8_t  magic;        /* 0xAB — sanity check, detect framing errors */
    uint8_t  version;      /* Protocol version: 0x01 = v1.0 */
    uint8_t  cmd;          /* Command opcode (OUT) or response opcode (IN) */
    uint8_t  flags;        /* Bit 0: more data follows; Bit 1: error flag */
    uint16_t sequence;     /* Sequence number — wraps at 0xFFFF */
    uint16_t length;       /* Length of payload following this header (bytes) */
} vendor_pkt_hdr_t;        /* Total: 8 bytes */

/* Command opcodes for bulk OUT */
#define CMD_PING             0x01  /* No payload — device echoes back */
#define CMD_GET_VERSION      0x02  /* No payload — device returns version string */
#define CMD_SET_LED          0x10  /* Payload: 1 byte (0=off, 1=on, 2=blink) */
#define CMD_READ_ADC         0x20  /* Payload: 1 byte (channel 0–7) */
#define CMD_START_STREAM     0x30  /* Payload: 4 bytes (sample_rate_hz) */
#define CMD_STOP_STREAM      0x31  /* No payload */

/* Response opcodes for bulk IN (high bit set to distinguish from commands) */
#define RSP_ACK              0x81  /* Generic acknowledgement */
#define RSP_NACK             0x82  /* Command rejected (reason in payload byte) */
#define RSP_VERSION          0x83  /* Payload: version string */
#define RSP_ADC_RESULT       0xA0  /* Payload: 2 bytes (uint16_t raw ADC value) */
#define RSP_STREAM_DATA      0xB0  /* Payload: N samples */

Interrupt IN Event Packet

/* Interrupt IN event packet — sent asynchronously by the device */
typedef struct __attribute__((packed)) {
    uint8_t  event_id;     /* Event type identifier */
    uint8_t  severity;     /* 0=info, 1=warning, 2=error, 3=critical */
    uint16_t timestamp_ms; /* Milliseconds since last reset (wraps) */
    uint8_t  data[4];      /* Event-specific data */
} vendor_event_t;          /* Total: 8 bytes, fits in one FS interrupt packet */

#define EVT_OVERRUN         0x01  /* Buffer overrun on ADC channel (data[0]=channel) */
#define EVT_THRESHOLD       0x02  /* Threshold crossed (data[0]=channel, data[1..2]=value) */
#define EVT_POWER_CHANGE    0x03  /* Supply voltage changed (data[0..1]=mV) */

WinUSB Descriptor for Driverless Windows

The single biggest obstacle to deploying a vendor-specific USB device is the Windows driver problem. On Linux and macOS, any USB device is accessible via libusb without any driver installation. On Windows, without a driver, the device shows up in Device Manager as "Unknown Device" and is inaccessible from user space.

WinUSB is a generic Windows kernel driver (winusb.sys) that exposes USB devices directly to user-mode applications via the WinUSB API. Once WinUSB is associated with your device, a user-mode application can open bulk and interrupt endpoints directly — without writing a kernel-mode driver. The key question is: how do you associate WinUSB with your device without requiring the user to install an INF file?

Microsoft OS 2.0 Descriptors (MSOS 2.0)

MSOS 2.0 (introduced with Windows 8.1) allows a USB device to embed driver-selection metadata in a special BOS (Binary device Object Store) descriptor. When Windows enumerates a device with a valid MSOS 2.0 descriptor, it automatically loads WinUSB — no INF file, no admin rights, no pop-up.

The mechanism uses a BOS descriptor with a Platform Capability Descriptor pointing to a Microsoft-defined GUID. The OS reads the BOS descriptor during enumeration, recognises the Microsoft Platform UUID, and sends a vendor GET request to retrieve the full MSOS 2.0 Descriptor Set.

/* === BOS Descriptor with MSOS 2.0 Platform Capability === */

/* Microsoft OS 2.0 Descriptor Set total length */
#define MSOS20_SET_TOTAL_LEN  (0x0A + 0x14 + 0x08 + 0x14)  /* = 72 bytes */

/* Vendor request code used to retrieve the MSOS 2.0 Descriptor Set */
#define VENDOR_REQ_MSOS20     0x01

/* Windows version: 0x06030000 = Windows 8.1 */
#define WIN_VERSION           0x06030000

/* WinUSB compatible ID GUID (placed in Feature Compatibly ID) */
#define WINUSB_COMPATIBLE_ID  "WINUSB\0\0"   /* 8 bytes, null-padded */

/* Full BOS Descriptor */
static const uint8_t bos_descriptor[] = {
    /* BOS Descriptor */
    0x05,                          /* bLength = 5 */
    0x0F,                          /* bDescriptorType = BOS */
    0x1D, 0x00,                    /* wTotalLength = 29 (BOS + one Platform Cap) */
    0x01,                          /* bNumDeviceCaps = 1 */

    /* Platform Capability Descriptor — Microsoft OS 2.0 */
    0x1C,                          /* bLength = 28 */
    0x10,                          /* bDescriptorType = Device Capability */
    0x05,                          /* bDevCapabilityType = Platform */
    0x00,                          /* bReserved */
    /* PlatformCapabilityUUID — Microsoft OS 2.0 Platform UUID (little-endian) */
    0xDF, 0x60, 0xDD, 0xD8,
    0x89, 0x45, 0xC7, 0x4C,
    0x9C, 0xD2, 0x65, 0x9D,
    0x9E, 0x64, 0x8A, 0x9F,
    /* dwWindowsVersion: 0x06030000 = Windows 8.1 */
    0x00, 0x00, 0x03, 0x06,
    /* wMSOSDescriptorSetTotalLength */
    (MSOS20_SET_TOTAL_LEN & 0xFF), (MSOS20_SET_TOTAL_LEN >> 8),
    /* bMS_VendorCode: request code to retrieve descriptor set */
    VENDOR_REQ_MSOS20,
    /* bAltEnumCode: 0 = no alternate enumeration */
    0x00
};

/* Full MSOS 2.0 Descriptor Set */
static const uint8_t msos20_descriptor_set[] = {
    /* Microsoft OS 2.0 Descriptor Set Header */
    0x0A, 0x00,                    /* wLength = 10 */
    0x00, 0x00,                    /* wDescriptorType = MS_OS_20_SET_HEADER_DESCRIPTOR */
    0x00, 0x00, 0x03, 0x06,        /* dwWindowsVersion = Windows 8.1 */
    (MSOS20_SET_TOTAL_LEN & 0xFF), (MSOS20_SET_TOTAL_LEN >> 8), /* wTotalLength */

    /* Configuration Subset Header (optional — use if device is not composite) */
    /* For single-interface device we go straight to Interface Subset */

    /* Microsoft OS 2.0 Compatible ID Descriptor */
    0x14, 0x00,                    /* wLength = 20 */
    0x03, 0x00,                    /* wDescriptorType = MS_OS_20_FEATURE_COMPATBLE_ID */
    /* CompatibleID: "WINUSB\0\0" */
    'W', 'I', 'N', 'U', 'S', 'B', 0x00, 0x00,
    /* SubCompatibleID: all zeros */
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,

    /* Microsoft OS 2.0 Registry Property Descriptor — DeviceInterfaceGUID */
    0x08, 0x00,                    /* wLength = 8 + name length + value length */
    /* NOTE: In practice wLength must include the full name and value UTF-16 strings.
       See the complete example below for the full structure with GUID. */
    0x04, 0x00,                    /* wDescriptorType = MS_OS_20_FEATURE_REG_PROPERTY */
    0x01, 0x00,                    /* wPropertyDataType = REG_SZ */
    0x28, 0x00                     /* wPropertyNameLength = 40 (in full descriptor) */
    /* Followed by UTF-16LE property name "DeviceInterfaceGUID\0" and value GUID */
};

Legacy WCID Method (Windows 7 and Earlier)

If you need Windows 7 support, the older WCID (Windows Compatible ID) method uses an OS String Descriptor at index 0xEE. Windows queries this string at address 0xEE; if it contains the magic signature "MSFT100" followed by a vendor request code, Windows will query the Compatible ID feature descriptor. This method still works on modern Windows but MSOS 2.0 is preferred for new designs.

Testing with USB Device Tree Viewer

After flashing firmware with MSOS 2.0 descriptors, use USB Device Tree Viewer (free, from Uwe Sieber) to inspect the BOS descriptor. Look for "Microsoft OS 2.0 Platform Capability Descriptor" in the device node. If absent, your BOS descriptor has an error. If present but WinUSB does not load, check that your vendor request handler returns the full descriptor set with the correct wIndex (0x0007).

Vendor Class Descriptor Assembly

The configuration descriptor for a vendor class device follows the same structure as any USB configuration descriptor — it is just the class codes that differ. Here is a complete vendor class configuration with Bulk IN, Bulk OUT, and optional Interrupt IN:

/* tusb_config.h additions for vendor class */
#define CFG_TUD_VENDOR              1       /* Enable one vendor class interface */
#define CFG_TUD_VENDOR_RX_BUFSIZE   512     /* Bulk OUT receive buffer (bytes) */
#define CFG_TUD_VENDOR_TX_BUFSIZE   512     /* Bulk IN transmit buffer (bytes) */

/* ------------------------------------------------------------------ */
/* Configuration descriptor for single vendor-class interface          */
/* ------------------------------------------------------------------ */

/* Endpoint numbers — must match tusb_config.h or your manual EP config */
#define EP_BULK_OUT     0x01   /* EP1 OUT */
#define EP_BULK_IN      0x81   /* EP1 IN  */
#define EP_INT_IN       0x82   /* EP2 IN  */

#define VENDOR_ITF_PROTOCOL  0x00  /* Your protocol identifier */
#define VENDOR_ITF_SUBCLASS  0x00  /* Your subclass identifier */

/* Full Speed endpoint max packet sizes */
#define EP_BULK_MPS     64
#define EP_INT_MPS      16
#define EP_INT_INTERVAL  5    /* 5 ms polling interval for interrupt IN */

/* Configuration descriptor total length:
   9 (config) + 9 (interface) + 7 (bulk out) + 7 (bulk in) + 7 (int in) = 39 */
#define CONFIG_TOTAL_LEN  39

static const uint8_t configuration_descriptor[] = {
    /* Configuration Descriptor */
    9,                          /* bLength */
    USB_DESC_TYPE_CONFIGURATION,/* bDescriptorType = 0x02 */
    CONFIG_TOTAL_LEN, 0x00,    /* wTotalLength */
    1,                          /* bNumInterfaces = 1 */
    1,                          /* bConfigurationValue = 1 */
    0,                          /* iConfiguration = no string */
    0x80,                       /* bmAttributes: bus-powered, no remote wakeup */
    250,                        /* bMaxPower: 500 mA (250 × 2 mA) */

    /* Interface Descriptor */
    9,                          /* bLength */
    USB_DESC_TYPE_INTERFACE,    /* bDescriptorType = 0x04 */
    0,                          /* bInterfaceNumber = 0 */
    0,                          /* bAlternateSetting = 0 */
    3,                          /* bNumEndpoints = 3 (bulk out, bulk in, int in) */
    0xFF,                       /* bInterfaceClass = Vendor-Specific */
    VENDOR_ITF_SUBCLASS,        /* bInterfaceSubClass */
    VENDOR_ITF_PROTOCOL,        /* bInterfaceProtocol */
    0,                          /* iInterface = no string */

    /* Endpoint Descriptor — Bulk OUT */
    7,                          /* bLength */
    USB_DESC_TYPE_ENDPOINT,     /* bDescriptorType = 0x05 */
    EP_BULK_OUT,                /* bEndpointAddress: EP1 OUT */
    0x02,                       /* bmAttributes: Bulk */
    EP_BULK_MPS, 0x00,          /* wMaxPacketSize */
    0,                          /* bInterval: ignored for bulk */

    /* Endpoint Descriptor — Bulk IN */
    7,                          /* bLength */
    USB_DESC_TYPE_ENDPOINT,
    EP_BULK_IN,                 /* bEndpointAddress: EP1 IN */
    0x02,                       /* bmAttributes: Bulk */
    EP_BULK_MPS, 0x00,
    0,

    /* Endpoint Descriptor — Interrupt IN */
    7,
    USB_DESC_TYPE_ENDPOINT,
    EP_INT_IN,                  /* bEndpointAddress: EP2 IN */
    0x03,                       /* bmAttributes: Interrupt */
    EP_INT_MPS, 0x00,
    EP_INT_INTERVAL             /* bInterval: 5 ms polling interval */
};

In TinyUSB, the tud_descriptor_configuration_cb() callback returns this byte array. The BOS descriptor is returned by tud_descriptor_bos_cb(). Both callbacks are mandatory if you are using MSOS 2.0.

TinyUSB Vendor Class Driver

TinyUSB ships with a built-in vendor class driver (src/class/vendor/vendor_device.c) that handles the bulk endpoint plumbing, leaving you to implement the protocol logic. With CFG_TUD_VENDOR 1 in tusb_config.h, the following API is available:

/* Check if there is data available to read from the bulk OUT endpoint */
bool tud_vendor_available(void);

/* Read up to bufsize bytes from the bulk OUT endpoint into buf.
   Returns the number of bytes actually read (may be less than bufsize). */
uint32_t tud_vendor_read(void* buf, uint32_t bufsize);

/* Write up to bufsize bytes to the bulk IN endpoint.
   Returns the number of bytes written (may be less than bufsize if TX buffer is full). */
uint32_t tud_vendor_write(void const* buf, uint32_t bufsize);

/* Flush the bulk IN transmit buffer — causes any buffered data to be sent immediately.
   Call this after the last tud_vendor_write() in a response sequence. */
uint32_t tud_vendor_write_flush(void);

Simple Command/Response Protocol Implementation

/* vendor_protocol.c — implements the custom command/response loop */
#include "tusb.h"
#include "vendor_protocol.h"

static uint8_t rx_buf[512];
static uint8_t tx_buf[512];

/* Called from your main loop or a dedicated USB task */
void vendor_protocol_task(void)
{
    if (!tud_vendor_available()) return;

    /* Read the packet header first */
    uint32_t n = tud_vendor_read(rx_buf, sizeof(rx_buf));
    if (n < sizeof(vendor_pkt_hdr_t)) return;  /* Discard undersized packets */

    vendor_pkt_hdr_t *hdr = (vendor_pkt_hdr_t *)rx_buf;

    /* Validate magic byte */
    if (hdr->magic != 0xAB) return;

    /* Prepare response header */
    vendor_pkt_hdr_t *rsp = (vendor_pkt_hdr_t *)tx_buf;
    rsp->magic    = 0xAB;
    rsp->version  = 0x01;
    rsp->flags    = 0x00;
    rsp->sequence = hdr->sequence;  /* Echo the sequence number */

    switch (hdr->cmd) {
        case CMD_PING: {
            rsp->cmd    = RSP_ACK;
            rsp->length = 0;
            tud_vendor_write(tx_buf, sizeof(vendor_pkt_hdr_t));
            break;
        }

        case CMD_GET_VERSION: {
            const char *ver = "usb-custom-driver v1.0.0";
            uint16_t vlen = (uint16_t)strlen(ver);
            rsp->cmd    = RSP_VERSION;
            rsp->length = vlen;
            memcpy(tx_buf + sizeof(vendor_pkt_hdr_t), ver, vlen);
            tud_vendor_write(tx_buf, sizeof(vendor_pkt_hdr_t) + vlen);
            break;
        }

        case CMD_SET_LED: {
            if (n < sizeof(vendor_pkt_hdr_t) + 1) {
                rsp->cmd    = RSP_NACK;
                rsp->length = 1;
                tx_buf[sizeof(vendor_pkt_hdr_t)] = 0x01; /* reason: bad length */
            } else {
                uint8_t led_state = rx_buf[sizeof(vendor_pkt_hdr_t)];
                board_led_control(led_state);
                rsp->cmd    = RSP_ACK;
                rsp->length = 0;
            }
            tud_vendor_write(tx_buf, sizeof(vendor_pkt_hdr_t) + rsp->length);
            break;
        }

        case CMD_READ_ADC: {
            if (n < sizeof(vendor_pkt_hdr_t) + 1) {
                rsp->cmd = RSP_NACK; rsp->length = 1;
                tx_buf[sizeof(vendor_pkt_hdr_t)] = 0x01;
            } else {
                uint8_t channel = rx_buf[sizeof(vendor_pkt_hdr_t)];
                uint16_t adc_val = adc_read_channel(channel);
                rsp->cmd    = RSP_ADC_RESULT;
                rsp->length = 2;
                tx_buf[sizeof(vendor_pkt_hdr_t) + 0] = (uint8_t)(adc_val & 0xFF);
                tx_buf[sizeof(vendor_pkt_hdr_t) + 1] = (uint8_t)(adc_val >> 8);
            }
            tud_vendor_write(tx_buf, sizeof(vendor_pkt_hdr_t) + rsp->length);
            break;
        }

        default:
            rsp->cmd    = RSP_NACK;
            rsp->length = 1;
            tx_buf[sizeof(vendor_pkt_hdr_t)] = 0xFF; /* reason: unknown command */
            tud_vendor_write(tx_buf, sizeof(vendor_pkt_hdr_t) + 1);
            break;
    }

    tud_vendor_write_flush();
}

/* Callback invoked by TinyUSB when a vendor control transfer is received on EP0 */
bool tud_vendor_control_xfer_cb(uint8_t rhport, uint8_t stage,
                                 tusb_control_request_t const *request)
{
    /* Only handle vendor-type, device-recipient requests */
    if ((request->bmRequestType & TUSB_REQ_TYPE_MASK)   != TUSB_REQ_TYPE_VENDOR)   return false;
    if ((request->bmRequestType & TUSB_REQ_RCPT_MASK)   != TUSB_REQ_RCPT_DEVICE)   return false;

    if (stage == CONTROL_STAGE_SETUP) {
        switch (request->bRequest) {
            case VENDOR_REQ_MSOS20:
                /* Return the MSOS 2.0 Descriptor Set when Windows requests it */
                if (request->wIndex == 0x0007) {
                    return tud_control_xfer(rhport, request,
                                            (void *)msos20_descriptor_set,
                                            sizeof(msos20_descriptor_set));
                }
                return false;

            case VENDOR_REQ_GET_PROTOCOL_VERSION: {
                static uint32_t version = 0x00010000; /* v1.0.0 */
                return tud_control_xfer(rhport, request,
                                        &version, sizeof(version));
            }

            case VENDOR_REQ_SET_SAMPLE_RATE:
                /* wValue contains the sample rate in Hz; wLength must be 0 */
                if (request->wLength == 0) {
                    adc_set_sample_rate(request->wValue);
                    return tud_control_status(rhport, request);
                }
                return false;

            default:
                return false;
        }
    }
    return true; /* Other stages (DATA, ACK) — TinyUSB handles them */
}

Writing a Reusable TinyUSB Class Driver

TinyUSB's vendor class is convenient but it is not a reusable class driver — it is a fixed single-interface implementation. If you want to distribute your protocol as a reusable component (e.g., a proprietary instrument class that other firmware teams can drop into their TinyUSB builds), you need to implement the usbd_class_driver_t interface.

The usbd_class_driver_t Structure

/* TinyUSB class driver interface — from src/device/usbd.h */
typedef struct {
    /* Name of this class driver (for debug output) */
    const char *name;

    /* Called once at startup to initialise driver state */
    void (*init)(void);

    /* Called on USB bus reset — reset all state machines */
    void (*reset)(uint8_t rhport);

    /* Called during enumeration when the host opens an interface.
       Return the number of bytes consumed from the descriptor or 0 to reject. */
    uint16_t (*open)(uint8_t rhport,
                     tusb_desc_interface_t const *itf_desc,
                     uint16_t max_len);

    /* Called when a class-specific control transfer arrives on EP0 */
    bool (*control_xfer_cb)(uint8_t rhport, uint8_t stage,
                             tusb_control_request_t const *request);

    /* Called when a bulk/interrupt/isochronous transfer completes */
    bool (*xfer_complete_cb)(uint8_t rhport, uint8_t ep_addr,
                             xfer_result_t result, uint32_t xferred_bytes);

    /* Called at the start of every USB frame (1 ms at FS) — optional, can be NULL */
    void (*sof)(uint8_t rhport, uint32_t frame_count);
} usbd_class_driver_t;

Registering Your Driver with TinyUSB

TinyUSB provides a weak callback usbd_app_driver_get_cb() that you implement to inject your custom class driver into the driver dispatch table:

/* myclass_device.c — reusable class driver implementation */
#include "tusb.h"
#include "myclass_device.h"

/* Driver state — one instance per class instance (extend for multiple) */
typedef struct {
    uint8_t itf_num;
    uint8_t ep_bulk_in;
    uint8_t ep_bulk_out;
    uint8_t ep_int_in;
} myclass_interface_t;

static myclass_interface_t _myclass_itf;

static void myclass_init(void) {
    memset(&_myclass_itf, 0, sizeof(_myclass_itf));
}

static void myclass_reset(uint8_t rhport) {
    (void)rhport;
    memset(&_myclass_itf, 0, sizeof(_myclass_itf));
}

static uint16_t myclass_open(uint8_t rhport,
                               tusb_desc_interface_t const *itf_desc,
                               uint16_t max_len)
{
    /* Only accept our specific class/subclass/protocol */
    TU_VERIFY(itf_desc->bInterfaceClass    == 0xFF, 0);
    TU_VERIFY(itf_desc->bInterfaceSubClass == VENDOR_ITF_SUBCLASS, 0);
    TU_VERIFY(itf_desc->bInterfaceProtocol == VENDOR_ITF_PROTOCOL, 0);

    _myclass_itf.itf_num = itf_desc->bInterfaceNumber;

    uint16_t drv_len = sizeof(tusb_desc_interface_t);
    uint8_t const *p_desc = tu_desc_next(itf_desc);

    /* Walk endpoint descriptors and open each one */
    for (uint8_t i = 0; i < itf_desc->bNumEndpoints; i++) {
        tusb_desc_endpoint_t const *ep_desc = (tusb_desc_endpoint_t const *)p_desc;
        TU_ASSERT(usbd_edpt_open(rhport, ep_desc), 0);

        uint8_t ep_addr = ep_desc->bEndpointAddress;
        if (tu_edpt_dir(ep_addr) == TUSB_DIR_IN) {
            if ((ep_desc->bmAttributes.xfer & 0x03) == TUSB_XFER_INTERRUPT)
                _myclass_itf.ep_int_in  = ep_addr;
            else
                _myclass_itf.ep_bulk_in = ep_addr;
        } else {
            _myclass_itf.ep_bulk_out = ep_addr;
            /* Prime the OUT endpoint to receive the first packet */
            usbd_edpt_xfer(rhport, ep_addr, rx_buf, sizeof(rx_buf));
        }

        drv_len += tu_desc_len(p_desc);
        p_desc   = tu_desc_next(p_desc);
    }

    return drv_len;
}

static bool myclass_xfer_cb(uint8_t rhport, uint8_t ep_addr,
                              xfer_result_t result, uint32_t xferred_bytes)
{
    (void)result;
    if (ep_addr == _myclass_itf.ep_bulk_out) {
        /* Data received from host — process it */
        myclass_handle_rx(rx_buf, xferred_bytes);
        /* Re-prime OUT endpoint for next packet */
        usbd_edpt_xfer(rhport, ep_addr, rx_buf, sizeof(rx_buf));
    }
    return true;
}

/* The static driver descriptor */
static const usbd_class_driver_t myclass_driver = {
    .name            = "MyVendorClass",
    .init            = myclass_init,
    .reset           = myclass_reset,
    .open            = myclass_open,
    .control_xfer_cb = tud_vendor_control_xfer_cb,
    .xfer_complete_cb= myclass_xfer_cb,
    .sof             = NULL
};

/* Weak override — TinyUSB calls this to get app-specific drivers */
usbd_class_driver_t const *usbd_app_driver_get_cb(uint8_t *driver_count)
{
    *driver_count = 1;
    return &myclass_driver;
}

Python libusb Host Application

The pyusb library provides a clean Pythonic interface to libusb-1.0 on Linux, macOS, and Windows (via WinUSB). Install it alongside the libusb backend:

pip install pyusb
# Windows: also install libusb-1.0 (from libusb.info) or use WinUSB backend
# Linux:   apt install libusb-1.0-0
# macOS:   brew install libusb

Complete Host Application

#!/usr/bin/env python3
"""
usb_custom_host.py — Host application for vendor USB class device
Communicates using the custom binary protocol defined in vendor_protocol.h
"""

import usb.core
import usb.util
import struct
import time

# Device identification
VENDOR_ID  = 0x1209   # pid.codes VID for open-source projects
PRODUCT_ID = 0x0001   # Your product ID

# Protocol constants
MAGIC       = 0xAB
VERSION     = 0x01
CMD_PING    = 0x01
CMD_VERSION = 0x02
CMD_SET_LED = 0x10
CMD_READ_ADC= 0x20
RSP_ACK     = 0x81
RSP_NACK    = 0x82
RSP_VERSION = 0x83
RSP_ADC     = 0xA0

HEADER_FMT  = '<BBBBHH'   # magic, version, cmd, flags, sequence, length
HEADER_SIZE = struct.calcsize(HEADER_FMT)

sequence_counter = 0

def build_packet(cmd, payload=b''):
    global sequence_counter
    sequence_counter = (sequence_counter + 1) & 0xFFFF
    hdr = struct.pack(HEADER_FMT,
                      MAGIC, VERSION, cmd, 0x00,
                      sequence_counter, len(payload))
    return hdr + payload

def parse_header(data):
    if len(data) < HEADER_SIZE:
        raise ValueError(f"Packet too short: {len(data)} bytes")
    magic, ver, cmd, flags, seq, length = struct.unpack_from(HEADER_FMT, data, 0)
    if magic != MAGIC:
        raise ValueError(f"Bad magic: 0x{magic:02X}")
    payload = data[HEADER_SIZE:HEADER_SIZE + length]
    return cmd, flags, seq, payload

def send_command(ep_out, ep_in, cmd, payload=b'', timeout=1000):
    """Send a command and wait for the response. Returns (rsp_cmd, payload)."""
    pkt = build_packet(cmd, payload)
    ep_out.write(pkt, timeout=timeout)
    rsp_data = ep_in.read(512, timeout=timeout)
    rsp_cmd, flags, seq, rsp_payload = parse_header(bytes(rsp_data))
    return rsp_cmd, rsp_payload

def main():
    # Find the device
    dev = usb.core.find(idVendor=VENDOR_ID, idProduct=PRODUCT_ID)
    if dev is None:
        raise RuntimeError("Device not found. Is it plugged in and enumerated?")

    print(f"Found device: VID=0x{VENDOR_ID:04X} PID=0x{PRODUCT_ID:04X}")
    print(f"Manufacturer: {usb.util.get_string(dev, dev.iManufacturer)}")
    print(f"Product:      {usb.util.get_string(dev, dev.iProduct)}")

    # Set active configuration
    dev.set_configuration()
    cfg = dev.get_active_configuration()

    # Get the interface
    intf = cfg[(0, 0)]

    # Find endpoints
    ep_out = usb.util.find_descriptor(intf,
        custom_match=lambda e:
            usb.util.endpoint_direction(e.bEndpointAddress) == usb.util.ENDPOINT_OUT and
            usb.util.endpoint_type(e.bmAttributes) == usb.util.ENDPOINT_TYPE_BULK)

    ep_in = usb.util.find_descriptor(intf,
        custom_match=lambda e:
            usb.util.endpoint_direction(e.bEndpointAddress) == usb.util.ENDPOINT_IN and
            usb.util.endpoint_type(e.bmAttributes) == usb.util.ENDPOINT_TYPE_BULK)

    if ep_out is None or ep_in is None:
        raise RuntimeError("Could not find bulk endpoints")

    print(f"EP OUT: 0x{ep_out.bEndpointAddress:02X}  EP IN: 0x{ep_in.bEndpointAddress:02X}")

    # --- Vendor control request: GET_PROTOCOL_VERSION ---
    version_bytes = dev.ctrl_transfer(
        bmRequestType=0xC0,        # IN | Vendor | Device
        bRequest=0x01,             # VENDOR_REQ_GET_PROTOCOL_VERSION
        wValue=0, wIndex=0,
        data_or_wLength=4)
    proto_ver = struct.unpack('<I', bytes(version_bytes))[0]
    print(f"Protocol version: 0x{proto_ver:08X}")

    # --- PING ---
    rsp_cmd, _ = send_command(ep_out, ep_in, CMD_PING)
    print(f"PING response: 0x{rsp_cmd:02X} ({'ACK' if rsp_cmd == RSP_ACK else 'NACK'})")

    # --- GET VERSION ---
    rsp_cmd, payload = send_command(ep_out, ep_in, CMD_VERSION)
    print(f"Firmware version: {payload.decode('ascii', errors='replace')}")

    # --- SET LED ---
    rsp_cmd, _ = send_command(ep_out, ep_in, CMD_SET_LED, bytes([1]))  # LED ON
    print(f"LED ON response: {'ACK' if rsp_cmd == RSP_ACK else 'NACK'}")

    # --- READ ADC channel 0 ---
    rsp_cmd, payload = send_command(ep_out, ep_in, CMD_READ_ADC, bytes([0]))
    if rsp_cmd == RSP_ADC and len(payload) >= 2:
        raw = struct.unpack_from('<H', payload, 0)[0]
        voltage = raw * 3.3 / 4095.0
        print(f"ADC ch0: raw={raw}  voltage={voltage:.3f} V")

    print("Done.")

if __name__ == '__main__':
    main()

Error Handling and Retry Logic

import usb.core

def send_with_retry(ep_out, ep_in, cmd, payload=b'', retries=3, timeout=500):
    """Send a command with automatic retry on timeout."""
    for attempt in range(retries):
        try:
            return send_command(ep_out, ep_in, cmd, payload, timeout=timeout)
        except usb.core.USBTimeoutError:
            print(f"Timeout on attempt {attempt+1}/{retries}")
            if attempt == retries - 1:
                raise
        except usb.core.USBError as e:
            print(f"USB error: {e}")
            raise
    return None

Custom Control Request Handling

USB control requests are the most precisely specified part of the protocol. Getting the bmRequestType field wrong is the most common error when implementing vendor control requests. Understanding the bit breakdown is essential:

Resulting common vendor request bmRequestType values:

• 0xC0 = IN | Vendor | Device — most common for GET requests
• 0x40 = OUT | Vendor | Device — for SET requests with no data return
• 0xC1 = IN | Vendor | Interface — for interface-specific GET requests
• 0x41 = OUT | Vendor | Interface — for interface-specific SET requests

/* Complete vendor control request handler — handles all vendor request types */
bool tud_vendor_control_xfer_cb(uint8_t rhport, uint8_t stage,
                                 tusb_control_request_t const *request)
{
    /* Only process vendor-type requests */
    if ((request->bmRequestType_bit.type) != TUSB_REQ_TYPE_VENDOR) return false;

    /* MSOS 2.0 — special case: device-directed, any vendor code */
    if (request->bRequest == VENDOR_REQ_MSOS20 &&
        request->wIndex   == 0x0007) {
        if (stage == CONTROL_STAGE_SETUP) {
            return tud_control_xfer(rhport, request,
                                    (void *)msos20_descriptor_set,
                                    sizeof(msos20_descriptor_set));
        }
        return true;
    }

    if (stage != CONTROL_STAGE_SETUP) return true;

    uint8_t recipient = request->bmRequestType_bit.recipient;

    if (recipient == TUSB_REQ_RCPT_DEVICE) {
        /* Device-directed vendor requests */
        switch (request->bRequest) {
            case VENDOR_REQ_GET_PROTOCOL_VERSION: {
                /* IN request: return 4-byte version */
                static const uint32_t ver = 0x00010000;
                return tud_control_xfer(rhport, request,
                                        (void *)&ver, sizeof(ver));
            }
            case VENDOR_REQ_GET_CAPABILITIES: {
                static const uint64_t caps =
                    (1ULL << 0) |   /* Bit 0: ADC present */
                    (1ULL << 1) |   /* Bit 1: LED present */
                    (1ULL << 4);    /* Bit 4: streaming supported */
                return tud_control_xfer(rhport, request,
                                        (void *)&caps, sizeof(caps));
            }
            case VENDOR_REQ_SET_SAMPLE_RATE:
                /* OUT, no data stage: wValue = rate in Hz */
                adc_set_sample_rate(request->wValue);
                return tud_control_status(rhport, request);

            case VENDOR_REQ_RESET_DEVICE:
                /* Schedule a soft reset after status stage */
                schedule_soft_reset();
                return tud_control_status(rhport, request);

            default:
                return false; /* Stall unsupported requests */
        }
    }

    if (recipient == TUSB_REQ_RCPT_INTERFACE) {
        /* Interface-directed vendor requests */
        switch (request->bRequest) {
            case VENDOR_REQ_GET_STATUS: {
                static uint32_t status_flags = 0;
                status_flags = build_status_flags();
                return tud_control_xfer(rhport, request,
                                        &status_flags, sizeof(status_flags));
            }
            default:
                return false;
        }
    }

    return false;
}

Testing and Protocol Debugging

Debugging a custom vendor class device requires a systematic approach. Unlike CDC or HID, there are no OS-level tools that understand your protocol — you must instrument both ends.

Wireshark Vendor Class Filter

Wireshark captures USB traffic via USBPcap (Windows) or usbmon (Linux). For vendor class devices, use these display filters:

# Show only vendor control requests (bmRequestType type=Vendor)
usb.bmRequestType.type == 0x02

# Show only bulk transfers on your device
usb.idVendor == 0x1209 and usb.transfer_type == 0x03

# Show the bulk OUT (host to device commands)
usb.idVendor == 0x1209 and usb.endpoint_address.direction == 0

# Show URB_BULK transfers (ignoring control, interrupt)
usb.urb_type == URB_BULK

Boundary Condition Testing

Your firmware must handle these edge cases correctly or it will fail in production:

• Maximum packet size: A bulk packet of exactly 64 bytes (Full Speed) must be handled without a buffer overflow. Many bugs appear only at exact multiples of the max packet size.
• Zero-length packet (ZLP): The USB spec requires a ZLP when a bulk transfer length is an exact multiple of the max packet size. If your host application sends 128 bytes (2 × 64), the firmware must handle the trailing ZLP correctly.
• Split packets: The host may split a logical packet across multiple USB transactions. Your firmware should not assume a full command arrives in a single tud_vendor_read() call.
• Rapid reconnect: Unplug and replug within 100 ms. Your state machine must reset cleanly without leaking state from the previous enumeration.

Loopback Test for Bulk Endpoints

/* loopback_test.c — echo all bulk OUT data back on bulk IN */
/* Useful as a first test to verify endpoint connectivity without protocol logic */

void vendor_loopback_task(void)
{
    if (!tud_vendor_available()) return;

    /* Read whatever arrived */
    uint32_t n = tud_vendor_read(rx_buf, sizeof(rx_buf));
    if (n == 0) return;

    /* Echo it back — may block if TX buffer is full */
    uint32_t sent = 0;
    while (sent < n) {
        uint32_t written = tud_vendor_write(rx_buf + sent, n - sent);
        sent += written;
        if (written == 0) tud_task(); /* Yield to USB stack if TX buffer is full */
    }
    tud_vendor_write_flush();
}

Protocol Fuzzing with Python

import random, struct

def fuzz_bulk_out(ep_out, ep_in, iterations=1000):
    """Send random data to the bulk OUT endpoint and check the device remains stable."""
    for i in range(iterations):
        # Random payload: 0 to 512 bytes
        size = random.randint(0, 512)
        data = bytes(random.getrandbits(8) for _ in range(size))
        try:
            ep_out.write(data, timeout=100)
            # Attempt to read a response — device may or may not respond
            try:
                ep_in.read(512, timeout=50)
            except:
                pass  # No response is OK for invalid commands
        except Exception as e:
            print(f"[{i}] USB error during fuzz: {e}")
            break
    # After fuzzing, verify device is still alive with a PING
    rsp_cmd, _ = send_command(ep_out, ep_in, CMD_PING)
    assert rsp_cmd == RSP_ACK, f"Device unresponsive after fuzzing: 0x{rsp_cmd:02X}"
    print(f"Fuzz complete ({iterations} iterations): device still alive")

Practical Exercises

Custom Driver Design Generator

Document your custom USB class driver design — project name, target MCU, VID/PID, endpoint plan, WinUSB requirement, and protocol notes. Download as Word, Excel, PDF, or PPTX for design review or project documentation.

Conclusion & Next Steps

Custom USB class drivers unlock the full flexibility of the USB protocol stack — at the cost of OS driver complexity and increased host application development effort. The key takeaways from this article:

• Use bInterfaceClass = 0xFF to declare a vendor-specific interface. Design your bInterfaceSubClass and bInterfaceProtocol values to uniquely identify your device family.
• MSOS 2.0 descriptors eliminate the Windows driver installation problem. A correctly implemented BOS descriptor with the Microsoft Platform UUID causes Windows to automatically bind WinUSB — zero user interaction required.
• Invest time in protocol header design upfront. A magic byte, version, sequence number, and length field in every bulk packet makes debugging and future evolution far easier.
• TinyUSB's vendor class (CFG_TUD_VENDOR) is the right starting point. For distributable libraries, implement usbd_class_driver_t and register via usbd_app_driver_get_cb().
• Python + pyusb is the fastest path to a working host application on all three OSes. libusb-1.0 handles the WinUSB/usbfs/IOKit backend transparently.
• Test boundary conditions explicitly: exact max-packet-size payloads, ZLP handling, rapid reconnect, and protocol fuzzing. These are the cases that fail in production if not tested in development.