Cannot compile YOLO11

Technical Support
1

Hi,

mx_nc --version

2.2.2

I want to compile the YOLO11 model I exported from Ultralytics using the Neural Compiler, but I’m encountering an error.

memryx.errors.OperatorError: Matmul (‘/model.10/m/m.0/attn/MatMul’) with axes=[3, 2] is not supported.

I’m generating the Ultralytics export as follows:

YOLO(“yolo11l.pt”).export(format=“onnx”, imgsz=640, simplify=True)

I’ve tried the YOLO11 medium, large, and small models, as well as the YOLO11 small pose model. I also tested with an imgsz of 480 and different offsets, but the result is still the same. How can I resolve this and get dfp file?

2

Hi @ahmet, thanks for bringing this to our attention!

As mentioned in our documentation on Neural Compiler Extensions, we support complex transformer blocks through compiler extensions. In this case, the issue is caused by a minor change in the attention subgraph pattern in YOLO11 models exported from newer Ultralytics releases (v8.4.82 and later).

To address this, our compiler team created a Neural Compiler Extension (.nce) that adds support for the updated YOLO11 attention pattern. Please download the extension file as follows and pass it to the compiler using the --extensions flag:

wget https://developer.memryx.com/example_files/Yolov26.nce

mx_nc -m your_model.onnx --autocrop --extensions path/to/Yolov26.nce

With this extension, the newer YOLO11 model variants should compile successfully. You might notice that the extension is named Yolov26.nce - this extension covers recent subgraph alterations made to YOLOv10, YOLO11, and YOLO26 variants in recent Ultralytics releases. The extension also maintains backward compatibility for earlier Ultralytics versions across all three model architectures.

We’re currently preparing a minor Neural Compiler release that will include this support out of the box. Once that version is available, you’ll simply be able to upgrade the compiler and compile these models without providing the extension separately. We’ll keep you posted when that release is available.

Sorry for the inconvenience, and thanks again for reporting the issue! Please let us know if you run into any other problems.

3

Hi,

Extension solved the problem, thanks.

I have one more question. I have a Memryx M2 form factor chip. According to the Compiler’s calculation, the large model requires 5 chips. I guess this won’t work with the device I have right now, right? I believe the M2 contains 4 chips, is that correct?

4

Hi @ahmet :waving_hand:

Yes to both. The M.2 form factor holds 4 MX3 chips (from the MemryX docs), and I checked the large model on my side too: it does not fit in 4. I compiled yolo11l/m/s straight from Ultralytics (export(format="onnx", imgsz=640), then mx_nc --autocrop -c 4) and this is what I got:

yolo11s (9.4M)  -> mapped to 4 chips  (compiler estimate ~200 fps at normal effort)
yolo11m (20M)   -> ResourceError: Resource Mapping failed: Please try using more chips
yolo11l (25M)   -> ResourceError: Resource Mapping failed (BaseFail:009). Please try using more chips

So on your 4-chip M.2 the detect variants that fit at 640 are small and nano. Medium and large run out of on-chip resources with 4 chips, which matches the 5-chip figure your compiler reported for large. The limit here is not the weights, it is the feature-map memory the head needs, so a bigger input also makes it worse. The docs mention chaining up to 16 chips as one logical device, so more than a single module is the path if you need m/l, but I can’t test the multi-module runtime here.

One extra thing, since you hit the attention error first: I ran this on mx_nc 2.2.5 and it compiled the YOLO11 attention block natively, no Yolov26.nce extension needed. That looks like the minor release the team mentioned in this thread. So if you upgrade the compiler you can drop the --extensions flag.

Hope this helps!

5

Hi @ahmet,

First off, thank you to @JESUSROYETH for sharing your advice and experience, and welcome to the community!

Yes, our M.2 module has 4 chips in total. At this time, we are unable support any YOLO11 large variants due to a restriction in the number of A-cores available in a 4-chip configuration. See the output below for more details on this failure point for YOLO11 large (640 input resolution).

. . . . . . . . . . . . . . . . . . . . 
Minimum Needed Core Resources:
	Total M-Cores:	223 / 768
	Total A-Cores:	444 / 384
	Max M-Cores per Tower:	1 / 32
. . . . . . . . . . . . . . . . . . . . 
Minimum Needed WMem Resources:
	Total WMem:	25063.94 / 43008 KByte
	Largest WMem per Layer:	1154.5 / 1792 KByte
. . . . . . . . . . . . . . . . . . . . 
Minimum Needed In/Out Ports:
	Input Ports:	1 / 32
	Output Ports:	6 / 32

Tip: If you’re curious about what type of resource error is occurring, you can run the compiler with maximum verbosity to see a more detailed output. Simply pass the -vvv flag to the compiler CLI.

As for YOLO11 medium variants, we do support the 320 and 480 resolutions, which can be referenced on our Model Explorer. The 640 variant will not compile to 4 chips because of insufficient feature map memory. To get around this issue, we have an MXA-optimized YOLO11 medium (640 res), where we replace the attention layers with convolution to reduce the required feature map memory. This optimized model can also be found on the Model Explorer (pre-compiled and trained on the standard 80 COCO classes). If you need a custom-trained model using this optimized architecture, feel free to follow up with us, and we can give you more details.

If you instead prefer to have more chips and skip the complications, we have a 16-chip PCIe card form factor coming soon, the Cascade 100P! Thanks :slight_smile: