CodeProject.AI Discussions

Re: CodeProjecvt.AI v2.5.4 , with Coral TPU and YOLOv8 Errors

26-Feb-24 2:01

Hi,

I'm running v2.5.4 with Object Detection (Coral) v2.14, for the life of me, I cannot get it working with YOLOv8. I'm getting the following all of the time, when it tries to detect something.

Terminal

12:51:35:Response rec'd from Object Detection (Coral) command 'detect' (...362187)
12:51:36:Object Detection (Coral):  [IndexError] : Traceback (most recent call last):
  File "C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\objectdetection_coral_adapter.py", line 167, in _do_detection
    result = do_detect(opts, img, score_threshold)
  File "C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\objectdetection_coral.py", line 222, in do_detect
    objs = detect.get_objects(interpreter, score_threshold, scale)
  File "C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\bin\windows\python39\venv\lib\site-packages\pycoral\adapters\detect.py", line 214, in get_objects
    elif common.output_tensor(interpreter, 3).size == 1:
  File "C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\bin\windows\python39\venv\lib\site-packages\pycoral\adapters\common.py", line 29, in output_tensor
    return interpreter.tensor(interpreter.get_output_details()[i]['index'])()
IndexError: list index out of range

I've tried changing the size of the model, to no avail.

Terminal

<pre>12:45:12:Server version:   2.5.4
12:45:15:
12:45:15:Module 'Object Detection (Coral)' 2.1.4 (ID: ObjectDetectionCoral)
12:45:15:Valid:         True
12:45:15:Module Path:   <root>\modules\ObjectDetectionCoral
12:45:15:AutoStart:     True
12:45:15:Queue:         objectdetection_queue
12:45:15:Runtime:       python3.9
12:45:15:Runtime Loc:   Local
12:45:15:FilePath:      objectdetection_coral_adapter.py
12:45:15:Pre installed: False
12:45:15:Start pause:   1 sec
12:45:15:Parallelism:   1
12:45:15:LogVerbosity:
12:45:15:Platforms:     all
12:45:15:GPU Libraries: installed if available
12:45:15:GPU Enabled:   enabled
12:45:15:Accelerator:
12:45:15:Half Precis.:  enable
12:45:15:Environment Variables
12:45:15:CPAI_CORAL_MODEL_NAME = YOLOv8
12:45:15:CPAI_CORAL_MULTI_TPU  = False
12:45:15:MODELS_DIR            = <root>\modules\ObjectDetectionCoral\assets
12:45:15:MODEL_SIZE            = medium
12:45:15:
12:45:15:Started Object Detection (Coral) module
12:45:17:Server: This is the latest version
12:45:22:objectdetection_coral_adapter.py: Using model yolov8, size medium
12:45:22:objectdetection_coral_adapter.py: TPU detected
12:45:22:objectdetection_coral_adapter.py: Using Edge TPU

I've only managed to get the MobileNet model working, but it is not very accurate even on large.

I've tried to run ..\..\setup again, as well

Thanks in advance for any help!

David

mailseth226-Feb-24 4:48

Re: CodeProjecvt.AI v2.5.4 , with Coral TPU and YOLOv8 Errors

26-Feb-24 4:48

Try enabling the multi-TPU option.

David Yip26-Feb-24 5:07

Re: CodeProjecvt.AI v2.5.4 , with Coral TPU and YOLOv8 Errors

26-Feb-24 5:07

Thanks! That seems to have fixed it, slightly counter intuitive, as I only have one USB TPU.

Are the following errors OK?

Terminal

15:55:11:System:           Windows
15:55:11:Operating System: Windows (Microsoft Windows 11 version 10.0.22621)
15:55:11:CPUs:             Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz (Intel)
15:55:11:                  1 CPU x 4 cores. 8 logical processors (x64)
15:55:11:GPU (Primary):    Intel(R) HD Graphics 530 (1,024 MiB) (Intel Corporation)
15:55:11:                  Driver: 27.20.100.9664
15:55:11:System RAM:       24 GiB
15:55:11:Platform:         Windows
15:55:11:BuildConfig:      Release
15:55:11:Execution Env:    Native
15:55:11:Runtime Env:      Production
15:55:11:.NET framework:   .NET 7.0.5
15:55:11:Default Python:
15:55:11:App DataDir:      C:\ProgramData\CodeProject\AI
15:55:11:Video adapter info:
15:55:11:  Intel(R) HD Graphics 530:
15:55:11:    Driver Version     27.20.100.9664
15:55:11:    Video Processor    Intel(R) HD Graphics Family
15:55:11:STARTING CODEPROJECT.AI SERVER
15:55:11:RUNTIMES_PATH             = C:\Program Files\CodeProject\AI\runtimes
15:55:11:PREINSTALLED_MODULES_PATH = C:\Program Files\CodeProject\AI\preinstalled-modules
15:55:11:MODULES_PATH              = C:\Program Files\CodeProject\AI\modules
15:55:11:PYTHON_PATH               = \bin\windows\%PYTHON_NAME%\venv\Scripts\python
15:55:11:Data Dir                  = C:\ProgramData\CodeProject\AI
15:55:11:Server version:   2.5.4
15:55:14:
15:55:14:Module 'Object Detection (Coral)' 2.1.4 (ID: ObjectDetectionCoral)
15:55:14:Valid:         True
15:55:14:Module Path:   <root>\modules\ObjectDetectionCoral
15:55:14:AutoStart:     True
15:55:14:Queue:         objectdetection_queue
15:55:14:Runtime:       python3.9
15:55:14:Runtime Loc:   Local
15:55:14:FilePath:      objectdetection_coral_adapter.py
15:55:14:Pre installed: False
15:55:14:Start pause:   1 sec
15:55:14:Parallelism:   1
15:55:14:LogVerbosity:
15:55:14:Platforms:     all
15:55:14:GPU Libraries: installed if available
15:55:14:GPU Enabled:   enabled
15:55:14:Accelerator:
15:55:14:Half Precis.:  enable
15:55:14:Environment Variables
15:55:14:CPAI_CORAL_MODEL_NAME = YOLOv8
15:55:14:CPAI_CORAL_MULTI_TPU  = True
15:55:14:MODELS_DIR            = <root>\modules\ObjectDetectionCoral\assets
15:55:14:MODEL_SIZE            = large
15:55:14:
15:55:14:Started Object Detection (Coral) module
15:55:16:Server: This is the latest version
15:55:21:objectdetection_coral_adapter.py: Using model yolov8, size large
15:55:21:objectdetection_coral_adapter.py: TPU detected
15:55:21:objectdetection_coral_adapter.py: Attempting multi-TPU initialisation
15:55:21:objectdetection_coral_adapter.py: Supporting multiple Edge TPUs
15:55:21:objectdetection_coral_adapter.py: WARNING: Logging before InitGoogleLogging() is written to STDERR
15:55:21:objectdetection_coral_adapter.py: I20240226 15:55:21.973263 14256 pipelined_model_runner.cc:171] Thread: 14256 receives empty request
15:55:21:objectdetection_coral_adapter.py: I20240226 15:55:21.973263 14256 pipelined_model_runner.cc:244] Thread: 14256 is shutting down the pipeline...
15:55:21:objectdetection_coral_adapter.py: I20240226 15:55:21.973263 14256 pipelined_model_runner.cc:254] Thread: 14256 Pipeline is off.
15:55:21:objectdetection_coral_adapter.py: I20240226 15:55:21.973263 13396 pipelined_model_runner.cc:206] Queue is empty and `StopWaiters()` is called.
15:55:21:objectdetection_coral_adapter.py: I20240226 15:55:21.973263 14256 pipelined_model_runner.cc:171] Thread: 14256 receives empty request
15:55:21:objectdetection_coral_adapter.py: E20240226 15:55:21.973263 14256 pipelined_model_runner.cc:239] Thread: 14256 Pipeline was turned off before.
15:55:21:objectdetection_coral_adapter.py: I20240226 15:55:21.975266 14256 pipelined_model_runner.cc:206] Queue is empty and `StopWaiters()` is called.
15:55:21:objectdetection_coral_adapter.py: E20240226 15:55:21.975266 14256 pipelined_model_runner.cc:239] Thread: 14256 Pipeline was turned off before.
15:55:21:objectdetection_coral_adapter.py: E20240226 15:55:21.975266 14256 pipelined_model_runner.cc:146] Failed to shutdown status: INTERNAL: Pipeline was turned off before.
15:55:27:Response rec'd from Object Detection (Coral) command 'detect' (...e3e58a) ['']  took 5543ms
15:55:27:Response rec'd from Object Detection (Coral) command 'detect' (...f3b589) ['']  took 346ms
15:55:28:Response rec'd from Object Detection (Coral) command 'detect' (...8ce805) ['']  took 342ms
15:55:28:Response rec'd from Object Detection (Coral) command 'detect' (...207b05) ['']  took 412ms
15:55:29:Response rec'd from Object Detection (Coral) command 'detect' (...c81b17) ['']  took 408ms
15:57:09:Response rec'd from Object Detection (Coral) command 'd

Thanks

mailseth226-Feb-24 5:48

Re: CodeProjecvt.AI v2.5.4 , with Coral TPU and YOLOv8 Errors

26-Feb-24 5:48

The multi-TPU code is basically a newer, different, code path. In theory it’s better, but it may also have more bugs and need to be matured a bit more. So it’s not the default.

I don’t see any problems in the above messages that you’re seeing.

FWIW, the USB connection tends to have its own flakey emergent properties. Sometimes it’s hard to get working reliably. Good to hear it’s working for you.

Also, I’d not recommend the large model with a single Coral TPU. Basically, the large YOLO model is 44 MB in size, but the TPU only contains 8 MB in cache, so most of the model runs on the CPU. You can get one or two more TPUs for better performance. With only one TPU, I’d run a small YOLO model.

David Yip26-Feb-24 5:58

CodeProject AI Detection Issues: Seeking Insights on Yolo.net vs CUDA Performance with Blue Iris

26-Feb-24 5:58

Thanks for the advice!

Zz0K226-Feb-24 0:08

Zz0K2

26-Feb-24 0:08

I'm struggling to understand what's happening and I require assistance. I had to opt for Yolo.net as the standard version that utilises CUDA consistently crashed after 12 hours. The discussion on this topic seems to have ceased. Nonetheless, could someone elucidate why, despite having quite aggressive trigger settings in Blue Iris (Night Profile: Min Confidence 60%, Pre-Trigger images: 3, Post-Trigger images: 30, Analyse one image every 100ms), CodeProject failed to detect me walking past my camera upon my return home? Here is the screenshot with AI analysis.

It keeps happening quite frequently now and when I was using CUDA (Yolo 5.6.2) I'm sure it was detecting and processing way better than this. Any ideas? Can I force the AI to recheck in this instance ?

It's important I address this as the image below could have well been an intruder !

Re: CodeProject AI Detection Issues: Seeking Insights on Yolo.net vs CUDA Performance with Blue Iris

Matthew Dennis27-Feb-24 4:26

Matthew Dennis

27-Feb-24 4:26

Could you please send a copy of the system info and the module info

"Mistakes are prevented by Experience. Experience is gained by making mistakes."

Re: CodeProject AI Detection Issues: Seeking Insights on Yolo.net vs CUDA Performance with Blue Iris

Zz0K227-Feb-24 7:57

Zz0K2

27-Feb-24 7:57

System Info:

Server version: 2.5.4
System: Windows
Operating System: Windows (Microsoft Windows 11 version 10.0.22631)
CPUs: Intel(R) Core(TM) i9-10850K CPU @ 3.60GHz (Intel)
1 CPU x 10 cores. 20 logical processors (x64)
GPU (Primary): NVIDIA GeForce GTX 1650 (4 GiB) (NVIDIA)
Driver: 522.30, CUDA: 11.8 (up to: 11.8), Compute: 7.5, cuDNN: 8.9
System RAM: 32 GiB
Platform: Windows
BuildConfig: Release
Execution Env: Native
Runtime Env: Production
.NET framework: .NET 7.0.10
Default Python:
Video adapter info:
NVIDIA GeForce GTX 1650:
Driver Version 31.0.15.2230
Video Processor NVIDIA GeForce GTX 1650
System GPU info:
GPU 3D Usage 24%
GPU RAM Usage 2.2 GiB
Global Environment variables:
CPAI_APPROOTPATH = <root>
CPAI_PORT = 32168

Module Info:

Module 'Object Detection (YOLOv5 .NET)' 1.9.3 (ID: ObjectDetectionYOLOv5Net)
Valid: True
Module Path: <root>\modules\ObjectDetectionYOLOv5Net
AutoStart: True
Queue: objectdetection_queue
Runtime: dotnet
Runtime Loc: Shared
FilePath: bin\ObjectDetectionYOLOv5Net.exe
Pre installed: False
Start pause: 1 sec
Parallelism: 0
LogVerbosity:
Platforms: all
GPU Libraries: installed if available
GPU Enabled: enabled
Accelerator:
Half Precis.: enable
Environment Variables
CUSTOM_MODELS_DIR = <root>\modules\ObjectDetectionYOLOv5Net\custom-models
MODELS_DIR = <root>\modules\ObjectDetectionYOLOv5Net\assets
MODEL_SIZE = medium
Module 'Object Detection (YOLOv5 .NET)' 1.9.3 (ID: ObjectDetectionYOLOv5Net)
Valid: True
Module Path: <root>\modules\ObjectDetectionYOLOv5Net
AutoStart: True
Queue: objectdetection_queue
Runtime: dotnet
Runtime Loc: Shared
FilePath: bin\ObjectDetectionYOLOv5Net.exe
Pre installed: False
Start pause: 1 sec
Parallelism: 0
LogVerbosity:
Platforms: all
GPU Libraries: installed if available
GPU Enabled: enabled
Accelerator:
Half Precis.: enable
Environment Variables
CUSTOM_MODELS_DIR = <root>\modules\ObjectDetectionYOLOv5Net\custom-models
MODELS_DIR = <root>\modules\ObjectDetectionYOLOv5Net\assets
MODEL_SIZE = medium
Module 'Object Detection (YOLOv5 .NET)' 1.9.3 (ID: ObjectDetectionYOLOv5Net)
Valid: True
Module Path: <root>\modules\ObjectDetectionYOLOv5Net
AutoStart: True
Queue: objectdetection_queue
Runtime: dotnet
Runtime Loc: Shared
FilePath: bin\ObjectDetectionYOLOv5Net.exe
Pre installed: False
Start pause: 1 sec
Parallelism: 0
LogVerbosity:
Platforms: all
GPU Libraries: installed if available
GPU Enabled: enabled
Accelerator:
Half Precis.: enable
Environment Variables
CUSTOM_MODELS_DIR = <root>\modules\ObjectDetectionYOLOv5Net\custom-models
MODELS_DIR = <root>\modules\ObjectDetectionYOLOv5Net\assets
MODEL_SIZE = medium
Status Data: {
"inferenceDevice": "GPU",
"inferenceLibrary": "DirectML",
"canUseGPU": true,
"successfulInferences": 132799,
"failedInferences": 0,
"numInferences": 132799,
"averageInferenceMs": 51,
"histogram": {
"person": 50202,
"dog": 2778,
"cat": 13447,
"bird": 918,
"pig": 4,
"vehicle": 124,
"horse": 1,
"car": 46,
"bear": 24,
"squirrel": 2,
"truck": 13,
"rabbit": 2,
"bus": 1,
"Person": 21,
"DayPlate": 2,
"Dog": 16
},
"numItemsFound": 67601
}
Started: 23 Feb 2024 4:59:16 PM GMT Standard Time
LastSeen: 27 Feb 2024 6:54:41 PM GMT Standard Time
Status: Started
Requests: 132804 (includes status calls)

Installation Log
2024-02-17 11:22:07: Installing CodeProject.AI Analysis Module
2024-02-17 11:22:07: ======================================================================
2024-02-17 11:22:07: CodeProject.AI Installer
2024-02-17 11:22:07: ======================================================================
2024-02-17 11:22:07: 795.0Gb of 952Gb available on System
2024-02-17 11:22:07: General CodeProject.AI setup
2024-02-17 11:22:07: Creating Directories...Done
2024-02-17 11:22:07: GPU support
2024-02-17 11:22:08: CUDA Present...Yes (CUDA 11.8, cuDNN 8.9)
2024-02-17 11:22:08: ROCm Present...No
2024-02-17 11:22:09: Reading ObjectDetectionYOLOv5Net settings.......Done
2024-02-17 11:22:09: Installing module Object Detection (YOLOv5 .NET) 1.9.3
2024-02-17 11:22:15: Downloading ObjectDetectionYOLOv5Net-DirectML-1.9.3.zip...Expanding...Done.
2024-02-17 11:22:15: Copying contents of ObjectDetectionYOLOv5Net-DirectML-1.9.3.zip to bin...done
2024-02-17 11:23:00: Downloading YOLO ONNX models...Expanding...Done.
2024-02-17 11:23:00: Copying contents of yolonet-models.zip to assets...done
2024-02-17 11:23:37: Downloading Custom YOLO ONNX models...Expanding...Done.
2024-02-17 11:23:37: Copying contents of yolonet-custom-models.zip to custom-models...done
2024-02-17 11:23:39: Self test: Self-test passed
2024-02-17 11:23:39: Module setup time 00:01:31.49
2024-02-17 11:23:39: Setup complete
2024-02-17 11:23:39: Total setup time 00:01:31.95
Installer exited with code 0

P.S. Had to install .NET version as CUDA kept failing every 10-12 hours with memory access issues as per this post:

CodeProject.AI Server: AI the easy way.[^]

Security?

WindowHero25-Feb-24 10:29

WindowHero

25-Feb-24 10:29

Maybe I'm missing something in the documentation, but I can't seem to find anything about any kind of authentication whatsoever. No passwords, no API keys, nothing. Given that this is clearly designed to be a network resource, has some resources to support that, and the documentation guides you to configurations where the server listens on wildcard (all addresses on the machine), how are you intended to secure it? Is it just assumed that your using namespaces and firewall rules to isolate it?

New Models not so good?

mpl133724-Feb-24 10:31

24-Feb-24 10:31

1:
V2.5.1 Single TPU Coral USB on Proxmox host -> passthrough to Container -> passthrough to Docker image:

V2.5.4 on Win11 with Dual TPU m.2 Adapter:

Why 2.5.1 detect more person?

2:

V2.5.1 Single TPU Coral USB on Proxmox host -> passthrough to Container -> passthrough to Docker image:

V2.5.4 on Win11 with Dual TPU m.2 Adapter:

Why doenst detect 2.5.4 the Person on the bicycle and why doesnt detect 2.5.1 the bicycle but the Person?

iv tryed all 3 Models on 2.5.4, but no changes.

BI detect on 2.5.4 my car as a sink and a vent pipe/flowerpot as a toilet. With 2.5.1 is the detection more precise.

here with 2.5.1

What can i do to get a better precision?

mailseth224-Feb-24 11:30

24-Feb-24 11:30

The MobileNet models are the oldest ones. Are you able to use the YOLO models? They are newer and should perform the best. Next I would experiment with using a model larger than small.

mpl133724-Feb-24 23:31

24-Feb-24 23:31

iv changed to Medium and Yolo, but why is on the Explorer Yolo5 and on the Module Setting Yolo8? whats correct?

must i change something on BLueiris too?

mailseth225-Feb-24 5:34

25-Feb-24 5:34

Looks like one of the numbers wasn’t changed in the interface. How is it working for you?

mpl133725-Feb-24 7:18

25-Feb-24 7:18

NOt really...

My car is a TV
My Heatpump is a car
My trash is a suitcase or a toilet

same picture on 2.5.1 and 2.5.4

2.5.1 looks good

and here with 2.5.4, not good

mailseth225-Feb-24 8:35

25-Feb-24 8:35

Could you post some of the originals so I can run some tests on them?

mpl133725-Feb-24 10:28

25-Feb-24 10:28

here is it

mailseth226-Feb-24 18:27

26-Feb-24 18:27

Here are all the models. cars_test.zip - Google Drive[^]

Notes:
All models were run on a Coral TPU with an 8-bit model.

The non-YOLO models were downloaded from the Coral example models page.

The input image has been stretched to fit the input tensor of the model. So 300x300 or 640x640, etc. One YOLOv8 model has an input that was stretched during model creation to already fit a landscape aspect ratio.

If the file is missing, nothing was found

David Yip4-Mar-24 6:04

4-Mar-24 6:04

I would tend to agree with you!

Using the v2.5.6 and a Coral with v2.1.4 - I have tried using Yolo, MobileNet and EfficientDet, I've had all sorts of results, ranging from Teddy Bear, TV, surfboard

I've gone back to using YOLLOv4 .NET on my CPU which is at least picking me up when I walk out of my front door, previously I wasn't getting anything

Show Coral TPUs

mpl133724-Feb-24 3:41