CodeProject.AI Discussions

21-Apr-24 11:36

My set up was CPAI on unraid as a container.
Windows on a sff dedicated pc (lenovo thinkcenter i5-3470T)

GPU GTX970 on unraid passed through to the container.


I had a few coral devices, A+E, M2 wifi and usb. Used on frigate

Tried the tpu on cpai last year and it failed miserably.

I have now a set up utilising the tpu with cpai and blueiris very well.

1- windows 10 X-Lite on the i5-3470T. 8GB ram. sata SSD
2- CPAI container on unraid (this can be CPAI on the windows pc also) - my sff has an A+E slot so may experiment.
3- Coral tpu m2 wifi on the asrock B550m board that hosts unraid. Used for my NAS.
4- AITool on the windows pc
5- BI Alert Tab - Confirm Alert with AI UNticked.
6- BI Record Tab - save jpegs to local folder on windows pc
7- AIool server pointing to unraid CPAI
8- AItool folder from the BI path for record jpegs

CPAI_CORAL_MODEL_NAME = YOLOv8
Size: Medium

Inference speeds: 8.6ms
round trips: 15-22ms

All sent to telegram and MQTT for nodered.

Removed my old GPU to reduce power consumption.

So far so good.

AITool was necessary here to finely tune each object (% image size etc)

Hope this helps someone struggling and wants to use a tpu.




12 cameras - 6 recording 24/7
CPU utilizes just 10-12%!

modified 23-Apr-24 12:49pm.

mailseth221-Apr-24 11:52

21-Apr-24 11:52

For what it’s worth, that sounds like it’s running YOLOv8 medium a bit too fast, so there may be a different model loaded up than expected. But if the results are good, no one’s gonna complain! Also, you should be able to plug in all of the TPUs and it’ll use them all simultaneously!

CP5216221-Apr-24 12:31

21-Apr-24 12:31

That's what I thought. It may well be using the MobileNet SSD Medium but the info says yolov8. Maybe a bit buggy still. Anyways it works and works well now.
I do have the USB tpu plugged in but I have that hosted for my frigate as a back up.
But its switched off.
May well play around with the multi-tpu. Thanks for the tip.
It is a bit unstable when you try to change settings, had to reinstall the module a couple of times.

mailseth221-Apr-24 12:50

21-Apr-24 12:50

Yeah. The downside of the USB one is that it may make the whole system less stable. Hard to say without trying it out, though. YOLOv8 medium works better on two than one, though, so I’d suggest giving it a try.

CP5216221-Apr-24 13:57

21-Apr-24 13:57

Nope it didn't play ball at all.

Added both devices to the container
/dev/apex_0
/dev/usb

Failed to work - even after reinstalling everything.

/dev/usb works on its own but it twice as slow but shows multi-tpu which is odd.
Removed that and readded /dev/apex_0 which as the fastest but would not work with the usb unfortunately.
Possibly if your board had slots for 2 pcie/wifi cards that might work but it did not like the usb and m2 adding to the container paths.

mailseth221-Apr-24 14:10

21-Apr-24 14:10

Huh. Weird. I see no reason why that wouldn’t work. I should just buy my own to test it. I currently have two M2 slots filled and three PCIe slots with dual TPU adapters.

CP5216221-Apr-24 14:35

21-Apr-24 14:35

Let me know how it goes if you test with multiple tpus.
May work if i try them on the dedicated windows pc directly rather than on unraid trying to pass them through to the container.

If I get time I'll try them on the windows (it has an A+E slot).

mailseth221-Apr-24 15:00

21-Apr-24 15:00

I have eight TPUs in my machine, just none of them USB. So I know it should work.

CP5216222-Apr-24 8:02

22-Apr-24 8:02

8??? omg. how does the multi-mode work then?

mailseth222-Apr-24 9:47

22-Apr-24 9:47

It works great on my machine! But developers always say that…

Last time I measured my per-TPU performance, it looked like this:
YOLO v8 issue with Coral TPU | IP Cam Talk[^]

CP5216221-Apr-24 14:43

21-Apr-24 14:43

Just a thought I wonder if the mesh would work like this. tpu on 1 instance of cpai and another instance of cpai with the 2nd tpu.

mailseth221-Apr-24 15:01

21-Apr-24 15:01

The pipelining between model segments will be more efficient if they are in the same machine.

CP5216221-Apr-24 15:11

21-Apr-24 15:11

ah nice - did not know unraid had the option in settings of running a 2nd instance of a container!

CP5216221-Apr-24 15:53

21-Apr-24 15:53

Wasn't able to get a 2nd instance running with a different port 32169 for example. It span up but somehow was linked to 32168 each time - even after making sure all the host ports were 32169 udp/tcp and web gui. unique name etc. But had no success

CP5216221-Apr-24 17:06

21-Apr-24 17:06

Set up a 2nd instance on the windows pc with the coral usb.
And 2 servers added on AITool so they share the 2 servers. Seems to work well and share the load.

mailseth221-Apr-24 17:49

21-Apr-24 17:49

I still think you should have better performance with one instance of CPAI running two TPUs, but if that doesn’t work, then I’m glad you got this working!

CP5216222-Apr-24 8:04