MX3 failed upgrade

Technical Support
1

Hi everyone,

I have an MX3 M.2 accelerator module (purchased a month plug ago from Mouser). It was working normally until I tried to upgrade the MemryX SDK. Below is the log. The accelerator no longer appear under lspci. I tried to slot it in another machine, still not detecting. Before this it ran great, I even ran a face recognition with nine 3k/4k streams and got 20-30FPS per stream. Now when I ran even one stream (using onnx model and not dfp) it barely reach 8FPS. Anyone has any idea how to make it work again?

memx_cascade_plus_pcie.ko.zst:

  • Uninstallation
    • Deleting from: /lib/modules/6.14.0-35-generic/updates/dkms/
  • Original module
    • No original module was found for this module on this kernel.
    • Use the dkms install command to reinstall any previous module version.
      depmod…
      Deleting module memx-drivers-2.0.5 completely from the DKMS tree.
      Unpacking memx-drivers (2.1.0-7.1) over (2.0.5-4.1) …
      Preparing to unpack …/5-memx-utils-gui_2.1.0-1_all.deb …
      Unpacking memx-utils-gui (2.1.0-1) over (2.0.0-1) …
      Preparing to unpack …/6-memx-utils-gui-cv46_2.1.0-7_amd64.deb …
      Unpacking memx-utils-gui-cv46 (2.1.0-7) over (2.0.0-8) …
      Setting up memx-utils-gui-cv46 (2.1.0-7) …
      Setting up memx-drivers (2.1.0-7.1) …
      Removing running module (if present)
      rmmod: ERROR: Module memx_cascade_plus_pcie is in use
      Analyzing connected MX3s…
      Flash booted MX3 found. Checking for firmware update…
      Update available! Now updating firmware on detected MX3 devices…

########## Memryx Flash UpdateTool - Directed (V1.6) ###############
0000:05:00.0 vendor=1fe9 device=0100 class=1200 irq=0 (pin 1) base0=dd000004 (De
vice 0100)
Found 1 MemryX Devices.

(DEICE1): Start Update Flash----
BAR0: Base Address: 0x00000000DD000000 Size=0x01000000
BAR1: Base Address: 0x0000000000000000 Size=0x00000000
BAR2: Base Address: 0x00000000DC000000 Size=0x01000000
BAR3: Base Address: 0x0000000000000000 Size=0x00000000
BAR4: Base Address: 0x00000000DE000000 Size=0x00100000
BAR5: Base Address: 0x0000000000000000 Size=0x00000000
Image size is: 0x1d978
Firmware update failed!
Building MX3 DKMS module…
Loading new memx-drivers-2.1.0 DKMS files…
Building for 6.14.0-35-generic
Building initial module for 6.14.0-35-generic
Done.

memx_cascade_usb.ko.zst:
Running module version sanity check.

  • Original module
    • No original module exists within this kernel
  • Installation
    • Installing to /lib/modules/6.14.0-35-generic/updates/dkms/

memx_cascade_plus_pcie.ko.zst:
Running module version sanity check.

  • Original module
    • No original module exists within this kernel
  • Installation
    • Installing to /lib/modules/6.14.0-35-generic/updates/dkms/
      depmod…
      NOTE! A system reboot is required!
      Setting up memx-accl (2.1.0-7) …
      Setting up memx-utils-gui (2.1.0-1) …
      Setting up mxa-manager (2.1.0-7) …
      Installing new version of config file /etc/memryx/mxa_manager.conf …
      Setting up memx-accl-plugins-d12 (2.1.0-7) …
      Setting up memx-accl-plugins (2.1.0-1) …
      Processing triggers for libc-bin (2.39-0ubuntu8.6) …
      office@office-NucBox-M6-Ultra:~$
2

Hello @ronne991 ,

From the log you shared, it appears that the firmware upgrade and the rmmod step both failed during the update process.

Could you please follow the steps below and let us know if the issue still persists?

  1. Purge all MemryX drivers and reinstall the latest version: Install Runtime (Linux) — MemryX Developer Hub

  2. Try updating the firmware independently from the SDK upgrade:: Update Firmware — MemryX Developer Hub

Also, could you please clarify the following statement?

“Now when I ran even one stream (using ONNX model and not DFP) it barely reaches 8 FPS.”

Are you referring to running the model directly through ONNX Runtime without compiling it to a DFP? Any additional details on how you’re running the inference would help us diagnose the performance issue.

3

Hi @sureshvairamuthu, I followed steps 1 and 2. Step 2 failed because the M.2 unit isn’t being detected.

��<�pre>########## Memryx Flash UpdateTool - Directed (V1.6) ############### Found 0 MemryX Devices. Memryx PCIe EP not found! <�/pre>

Yes, that’s what I meant. I didn’t use DFP because the unit isn’t being detected. I used DFP before this issue. Now I am only running the program with onnx

4

Hi everyone,

I hit the same issue when running software update on Ubuntu. It hangs on the firmware update process ( I waited for 2 hours and no status update…). Then, the firmware update failed.

After that, I tried to re-run the firmware update command on terminal. The process does not finish. I have been waiting for another 30 minutes. Is there any hint to fix this problem?

Thanks.

((venv) ) linaro@rk3588-ubuntu:~$ sudo /usr/sbin/mxfw_pcie_update_flash -f /lib/firmware/cascade_4chips_flash.bin

########## Memryx Flash UpdateTool - Directed (V1.6) ###############
0000:01:00.0 vendor=1fe9 device=0100 class=1200 irq=0 (pin 1) base0=900000004 (Device 0100)
Found 1 MemryX Devices.

(DEICE1): Start Update Flash----
BAR0: Base Address: 0x0000000900000000 Size=0x01000000
BAR1: Base Address: 0x0000000000000000 Size=0x00000000
BAR2: Base Address: 0x0000000901000000 Size=0x01000000
BAR3: Base Address: 0x0000000000000000 Size=0x00000000
BAR4: Base Address: 0x0000000902000000 Size=0x00100000
BAR5: Base Address: 0x0000000000000000 Size=0x00000000
Image size is: 0x1d978

5

After reboot, the flash image upgrade finished in 1 sec….:slightly_smiling_face:

1 Like