Subject: [ANNOUNCE] mthca: alternative Mellanox HCA driver
Because of the large number of requests for a simpler HCA driver, I
decided to clean up my "science project" driver for the Mellanox HCA
and release it. The driver is extremely immature but does enough for
IPoIB to work. I created a branch of the subversion repository in
, and this new "mthca"
driver code is in the directory
This driver currently cannnot be used as a full replacement for
Mellanox's driver, as it is quite incomplete and does not support many
critical features (more details below).
However, this driver was written from scratch as a fully native Linux
2.6 driver (in fact, because CLEAR_BITMAP was for some reason renamed
to bitmap_zero, you will need to build against Linux 2.6.6 or newer).
I believe that the driver can serve at least as a good proof of
concept and should clarify the OpenIB driver architecture. The stack
with the mthca low-level driver looks like:
ib_core etc. (ib_mad, ib_client_query etc. will be gradually
merged into a single ib_core module):
The core IB midlayer.
A standard Linux PCI driver for the Mellanox HCA, which plugs
ib_ipoib, ib_sdp, ib_srp, etc.:
Upper-level protocol drivers that use the ib_core API to
provide useful protocols over IB.
The mthca driver currently does _far_ less than Mellanox's mature HCA
driver. The following list of limitations is probably incomplete but
should give some idea of the level of completeness:
No userspace support: ib_mthca provides _no_ facilities for
ss to the HCA, beyond the generic MAD services
provided by the ib_useraccess module.
Firmware commands are polled: ib_mthca does not implement
event-driven firmware commands (which limits it to one
outstanding command at a time).
Only UD transport: ib_mthca does not handle transports other than
UD correctly, so the only functional ULP is IPoIB.
No FMR support: ib_mthca does not support Mellanox-style "fast
memory regions" as used by SDP and SRP.
No SRQ support: ib_mthca does not support shared receieve queues.
Only handles successful completions: the code to process
completions with an error status is not written yet.
Limited support for asynchronous events: only port change
asynchronous events will be dispatched.
Only physical memory registration: the register virtual memory
region verb is not implemented.
UD address vectors are in host memory: putting address vectors in
HCA memory increases performance; ib_mthca does not implement
Comments, questions and discussion are very welcome. I'm happy to get
even the most nitpicking suggestions on the code.
To unsubscribe send an email with subject unsubscribe to
Please contact [email protected]