Frigate on TrueNAS

Frigate
1

I built a solution to allow you to use MemryX M2 on TruenNAS with Frigate
It uses sysext package to layer needed driver and daemon

it can be found here

its a shame MemryX requires the container to be privileged compares to competring accelerators that do not, this needs to be looked at.

2

Awesome work, glad to see you got it running on TrueNAS! :tada:

Regarding privileged, we are working on an improvement to our Frigate integration, which among other things will remove the need for privileged.

We’re adding an MX3 runner for face recognition models, so it can be used for both the detector and recognition simultaneously. We’ll switch from ‘local mode’ to ‘shared mode’, which will:

  1. Allow multiple models (detection + recognition) to run simultaneously.
  2. Remove the need for privileged. The device won’t need to be passed either, since all send/recv will happen over the mxa_manager socket.
  3. Loosen the SDK version requirements between container and host. The driver library (libmemx.so) and runtime (libmx_accl.so) change frequently, while the packet protocol between apps and mxa_manager usually stays backwards-compatible. So for example, it will be okay if the container image still uses SDK 2.1 runtime while the host has SDK 2.2 driver & manager.

The initial Frigate integration used Local because we were only running the detection model, and the data copy over the mxa_manager socket added overhead that slowed down weak host CPUs like Raspberry Pi. This overhead is still there, but we’ve improved the performance a lot in SDK 2.3 and consider the pros to outweigh the cons. But we also value community feedback, so let me know if you have concerns!

3

at @mx-tim that awesome, excited to see where you go with this, yeah developing sysexts to add arbitrary hardware support to truenas has been interesting :slight_smile:

any chance of removing this noise in dmesg - it seems to happens when frigate holds open the hailo but there is no inferencing happening because frigate is not sending anything (because nothing walked in front of the cameras, lol. Or maybe this is frigate not using the device correctly (eg forcing exclusive mode?)

i think your device is great, your open source approach to firmware and software made the sysext way easier to do (i basically oneshotted it with Fable model on friday before it was turned off) can’t wait to see the improvements :slight_smile:

[ +9.262913] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239889] memryx: fops_read: wait timeout 10(s), retrying again
[Jun13 19:56] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239878] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239872] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.243885] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.235905] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239861] memryx: fops_read: wait timeout 10(s), retrying again
[Jun13 19:57] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.243853] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.235889] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239902] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239886] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239883] memryx: fops_read: wait timeout 10(s), retrying again
[Jun13 19:58] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239874] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.243884] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.235886] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239879] memryx: fops_read: wait timeout 10(s), retrying again
[Jun13 19:59] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239886] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239880] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239887] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239909] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239864] memryx: fops_read: wait timeout 10(s), retrying again
[Jun13 20:00] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.243860] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.235889] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239925] memryx: fops_read: wait timeout 10(s), retrying again
[ +10.239849] memryx: fops_read: wait timeout 10(s), retrying again
[ +8.211919] usb 4-2: USB disconnect, device number 11

4

forgot, no issues with the changes you outlined, seem reasonable to me, though i already submitted a PR to the truenas frigate app maintainer to add priviged as a flag in the interim

frigate: add Privileged Mode toggle by scyto · Pull Request #5164 · truenas/apps

5

and to spam you again, meh, i will patch the driver to remove the fops read spam behind a #ifdef DEBUG

if it works i will PR it and you can take it as you see fit/not fit :slight_smile:

–edit-- 41m later :slight_smile:
done kdriver: gate fops_read/fops_write timeout logs behind #ifdef DEBUG by scyto · Pull Request #2 · memryx/mx3_driver_pub

tested x64 version locally
and full disclosure i used claude to do the anlaysis and work at my direction