|
Here's what I'm seeing from the startup logs, nothing jumps out as an error:
10:06:52 AM: App directory C:\Program Files\CodeProject\SenseAI
10:06:52 AM: Started Background Removal
10:06:52 AM: Not starting Legacy Object Detection: Not set as enabled
10:06:52 AM: Started Portrait Filter
10:06:52 AM: Started Sentiment Analysis
10:06:53 AM: PortraitFilter.dll: Starting inference session...
10:06:54 AM: PortraitFilter.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: PortraitFilter.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: PortraitFilter.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: ObjectDetector.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: ObjectDetector.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: ObjectDetector.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: SentimentAnalysis.dll: 2022-06-21 10:06:54.821194: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: C:\Program Files\CodeProject\SenseAI\AnalysisLayer\SentimentAnalysis\sentiment_model
10:06:54 AM: SentimentAnalysis.dll: 2022-06-21 10:06:54.827749: I tensorflow/cc/saved_model/reader.cc:148] Reading SavedModel debug info (if present) from: C:\Program Files\CodeProject\SenseAI\AnalysisLayer\SentimentAnalysis\sentiment_model
10:06:54 AM: SentimentAnalysis.dll: To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
10:06:54 AM: SentimentAnalysis.dll: 2022-06-21 10:06:54.896526: I tensorflow/cc/saved_model/loader.cc:283] SavedModel load for tags { serve }; Status: success: OK. Took 75320 microseconds.
10:06:54 AM: scene.py: Scene Detection module started.
10:06:55 AM: Latest version available is 1.4-Beta
10:06:55 AM: Version check: This is the latest version
10:06:55 AM: PortraitFilter.dll: Background Portrait Filter Task Started.
10:06:55 AM: SentimentAnalysis.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:55 AM: SentimentAnalysis.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:55 AM: SentimentAnalysis.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:55 AM: SenseAI Object Detection module started.
10:06:55 AM: face.py: Face Detection module started.
10:06:56 AM: sense_rembg_adapter.py: RemoveBackground module started.
10:06:56 AM: textsummary.py: TextSummary module started.
10:06:56 AM: SentimentAnalysis.dll: info: SentimentAnalysis.SentimentAnalysisWorker[0]
10:06:56 AM: Sentiment Analysis module started.
Also, just to be clear, face recognition is working great (and fast!) for me:
piece = <PIL Image Object>
buf = io.BytesIO()
piece.save(buf, format='JPEG')
byte_im = buf.getvalue()
response = requests.post("http://daedelus.lan:5000/v1/vision/detection", files={"image": byte_im}).json()
for object in response["predictions"]:
print("{} - {}%".format(object["label"], round(object["confidence"]*100, 1)))
print(response)
response = requests.post("http://daedelus.lan:5000/v1/vision/face", files={"image": byte_im}).json()
print('FACE finder: {}'.format(response))
for object in response["predictions"]:
face_boxes = [object["x_min"], object["y_min"], object["x_max"], object["y_max"]]
print("{} - {}%".format(face_boxes, round(object["confidence"]*100, 1)))
face = piece.copy().crop((face_boxes[0], face_boxes[1], face_boxes[2], face_boxes[3]))
face.save(buf, format='JPEG')
byte_im = buf.getvalue()
response = requests.post("http://daedelus.lan:5000/v1/vision/face/recognize", files={"image": byte_im}).json()
print('FACE recognizer: {}'.format(response))
for object_f in response["predictions"]:
print("FACE: {} - {}%".format(object_f["userid"], round(object_f["confidence"]*100, 1)))
retults:
person - 92.3%
tennis racket - 38.7%
car - 92.5%
potted plant - 34.1%
tennis racket - 52.7%
{'predictions': [{'label': 'person', 'confidence': 0.9226199, 'y_min': 662, 'x_min': 547, 'y_max': 1522, 'x_max': 1025}, {'label': 'tennis racket', 'confidence': 0.38740522, 'y_min': 793, 'x_min': 538, 'y_max': 895, 'x_max': 771}, {'label': 'car', 'confidence': 0.924697, 'y_min': 29, 'x_min': 17, 'y_max': 509, 'x_max': 820}, {'label': 'potted plant', 'confidence': 0.34054393, 'y_min': 1353, 'x_min': 393, 'y_max': 1550, 'x_max': 652}, {'label': 'tennis racket', 'confidence': 0.5270328, 'y_min': 897, 'x_min': 270, 'y_max': 1260, 'x_max': 808}], 'success': True}
FACE finder: {'success': True, 'predictions': [{'confidence': 0.6121846437454224, 'x_min': 829, 'y_min': 682, 'x_max': 890, 'y_max': 778}]}
[829, 682, 890, 778] - 61.2%
FACE recognizer: {'success': True, 'predictions': [{'confidence': 0.6947708129882812, 'userid': 'Jasmine', 'x_min': 829, 'y_min': 682, 'x_max': 890, 'y_max': 778}]}
FACE: Jasmine - 69.5%
It's just not returning the face information through BlueIris when I point BI at the SenseAI url. It is returning all the results of the detection predictions. I don't have enough insight to see what BI is calling internally and whether the APIs match.
Thanks again for your attention!
|
|
|
|
|
Hey guys congrats on the project it's excellent.
Seems it's fully compatible with Agent DVR - might want to add that to the compatibility docs with a setup walkthrough?
Agent DVR DeepStack AI/ senseAI[^]
|
|
|
|
|
That's fantastic! I'll try it out and add it to the docs.
Thanks for the tip. 👍
cheers
Chris Maunder
|
|
|
|
|
|
Funny you should mention that! We were talking about exactly this today and I'd like it to be done this week.
I have a personal need of a racoon detector. It's a long, painful story.
cheers
Chris Maunder
|
|
|
|
|
Great, I also am working on a Python rest api that uses DeepStack to do ALPR, do you think there is a way to add it to this project. I am not a programmer but with Google I was able to make it work.
|
|
|
|
|
I see 365 scene types and 80 objects can be detected. Is there a list of detectable scenes and objects?
|
|
|
|
|
I think they are using YOLOv5 models which has the below objects
'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',
'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',
'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',
'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone',
'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear',
'hair drier', 'toothbrush'
|
|
|
|
|
I don't see the keyword "package". That's a bummer for my camera watching the front porch.
But hey... Detecting a toilet can be good.
|
|
|
|
|
Yeah - the amazon package delivery is a custom model we will add (or even better: source from someone kind!)
It's high on my TODO list.
cheers
Chris Maunder
|
|
|
|
|
Chris,
I could look into making a package model or add it to one of my custom models.
Mike
|
|
|
|
|
Yes please!
cheers
Chris Maunder
|
|
|
|
|
What model format are you using from what I can tell it is ONNX?
|
|
|
|
|
We're going to lean on Deepstack's model training, which is essentially the YOLO training (same code more or less) which outputs PyTorch .pt files.
cheers
Chris Maunder
|
|
|
|
|
Great, that is how all my models are trained. I just trained a package model, it only took 10 minutes on my RTX 3090. I will see how it performs before I added it to my other models.
|
|
|
|
|
Nice! Let us know how it goes.
cheers
Chris Maunder
|
|
|
|
|
Do you have your models posted somewhere? Or a quick walkthrough (or pointer) for training a model and putting it in the right format/place?
I've got all my security cameras wired up and sending alerts and it's awesome... next step is having a "package on the porch" model or a "black car is in driveway" model or a "fence gate is open" model, etc. I use Home Assistant, so the bonus is then I can have those models fire and tie them to sensors that I can then automate (e.g. "If Gate is open and time is after sundown, turn on back yard lights and send me a notification with picture")
|
|
|
|
|
|
Why not look at YOLOv5 it is more advanced. I think DeepStack was ported from YOLOv5 V3.1 and currently YOLOv5 is at V6.1. If you do go with YOLOv5 the DeepStack models will need to be retrained.
|
|
|
|
|
That's absolutely where we're headed. DeepStack was an easy integration but it does have a lot of legacy baggage we want to clean out, and our feeling is simply replacing with the original code that they themselves used is probably the path of least pain.
cheers
Chris Maunder
|
|
|
|
|
I did find a package dataset, it is a small dataset, it only has 250 images. I am not sure how well it will train if I add it to one of my dataset because the datasets I am using for my custom models have over 10,000 images
Packages Object Detection Dataset - augmented-v1[^]
|
|
|
|
|
It's always worth trying out what we can, and then we make it better over time.
cheers
Chris Maunder
|
|
|
|
|
Hi Mike,
I noticed that Blue Iris sometimes flags objects as "cars" while other time as a "vehicle" - is vehicle a value label in your model? Or is this because "Mark As Vehicle" is enabled and thus Blue Iris groups multiple tags together under the vehicle label?
UPDATE: Nevermind, just saw there is a new Github page for the Sense AI versions of the models:
CodeProject.AI-Custom-IPcam-Models
IPcam-combined Labels: - person, bicycle, car, motorcycle, bus, truck, bird, cat, dog, horse, sheep, cow, bear, deer, rabbit, raccoon, fox, skunk, squirrel, pig
IPcam-general Labels (includes dark models images): - person, vehicle
IPcam-animal Labels: - bird, cat, dog, horse, sheep, cow, bear, deer, rabbit, raccoon, fox, skunk, squirrel, pig
IPcam-dark Labels: - Bicycle, Bus, Car, Cat, Dog, Motorcycle, Person
modified 8-Oct-22 15:05pm.
|
|
|
|
|
I was just going to post the exact same model details.
|
|
|
|
|
The scenes are
/a/airfield 0
/a/airplane_cabin 1
/a/airport_terminal 2
/a/alcove 3
/a/alley 4
/a/amphitheater 5
/a/amusement_arcade 6
/a/amusement_park 7
/a/apartment_building/outdoor 8
/a/aquarium 9
/a/aqueduct 10
/a/arcade 11
/a/arch 12
/a/archaelogical_excavation 13
/a/archive 14
/a/arena/hockey 15
/a/arena/performance 16
/a/arena/rodeo 17
/a/army_base 18
/a/art_gallery 19
/a/art_school 20
/a/art_studio 21
/a/artists_loft 22
/a/assembly_line 23
/a/athletic_field/outdoor 24
/a/atrium/public 25
/a/attic 26
/a/auditorium 27
/a/auto_factory 28
/a/auto_showroom 29
/b/badlands 30
/b/bakery/shop 31
/b/balcony/exterior 32
/b/balcony/interior 33
/b/ball_pit 34
/b/ballroom 35
/b/bamboo_forest 36
/b/bank_vault 37
/b/banquet_hall 38
/b/bar 39
/b/barn 40
/b/barndoor 41
/b/baseball_field 42
/b/basement 43
/b/basketball_court/indoor 44
/b/bathroom 45
/b/bazaar/indoor 46
/b/bazaar/outdoor 47
/b/beach 48
/b/beach_house 49
/b/beauty_salon 50
/b/bedchamber 51
/b/bedroom 52
/b/beer_garden 53
/b/beer_hall 54
/b/berth 55
/b/biology_laboratory 56
/b/boardwalk 57
/b/boat_deck 58
/b/boathouse 59
/b/bookstore 60
/b/booth/indoor 61
/b/botanical_garden 62
/b/bow_window/indoor 63
/b/bowling_alley 64
/b/boxing_ring 65
/b/bridge 66
/b/building_facade 67
/b/bullring 68
/b/burial_chamber 69
/b/bus_interior 70
/b/bus_station/indoor 71
/b/butchers_shop 72
/b/butte 73
/c/cabin/outdoor 74
/c/cafeteria 75
/c/campsite 76
/c/campus 77
/c/canal/natural 78
/c/canal/urban 79
/c/candy_store 80
/c/canyon 81
/c/car_interior 82
/c/carrousel 83
/c/castle 84
/c/catacomb 85
/c/cemetery 86
/c/chalet 87
/c/chemistry_lab 88
/c/childs_room 89
/c/church/indoor 90
/c/church/outdoor 91
/c/classroom 92
/c/clean_room 93
/c/cliff 94
/c/closet 95
/c/clothing_store 96
/c/coast 97
/c/cockpit 98
/c/coffee_shop 99
/c/computer_room 100
/c/conference_center 101
/c/conference_room 102
/c/construction_site 103
/c/corn_field 104
/c/corral 105
/c/corridor 106
/c/cottage 107
/c/courthouse 108
/c/courtyard 109
/c/creek 110
/c/crevasse 111
/c/crosswalk 112
/d/dam 113
/d/delicatessen 114
/d/department_store 115
/d/desert/sand 116
/d/desert/vegetation 117
/d/desert_road 118
/d/diner/outdoor 119
/d/dining_hall 120
/d/dining_room 121
/d/discotheque 122
/d/doorway/outdoor 123
/d/dorm_room 124
/d/downtown 125
/d/dressing_room 126
/d/driveway 127
/d/drugstore 128
/e/elevator/door 129
/e/elevator_lobby 130
/e/elevator_shaft 131
/e/embassy 132
/e/engine_room 133
/e/entrance_hall 134
/e/escalator/indoor 135
/e/excavation 136
/f/fabric_store 137
/f/farm 138
/f/fastfood_restaurant 139
/f/field/cultivated 140
/f/field/wild 141
/f/field_road 142
/f/fire_escape 143
/f/fire_station 144
/f/fishpond 145
/f/flea_market/indoor 146
/f/florist_shop/indoor 147
/f/food_court 148
/f/football_field 149
/f/forest/broadleaf 150
/f/forest_path 151
/f/forest_road 152
/f/formal_garden 153
/f/fountain 154
/g/galley 155
/g/garage/indoor 156
/g/garage/outdoor 157
/g/gas_station 158
/g/gazebo/exterior 159
/g/general_store/indoor 160
/g/general_store/outdoor 161
/g/gift_shop 162
/g/glacier 163
/g/golf_course 164
/g/greenhouse/indoor 165
/g/greenhouse/outdoor 166
/g/grotto 167
/g/gymnasium/indoor 168
/h/hangar/indoor 169
/h/hangar/outdoor 170
/h/harbor 171
/h/hardware_store 172
/h/hayfield 173
/h/heliport 174
/h/highway 175
/h/home_office 176
/h/home_theater 177
/h/hospital 178
/h/hospital_room 179
/h/hot_spring 180
/h/hotel/outdoor 181
/h/hotel_room 182
/h/house 183
/h/hunting_lodge/outdoor 184
/i/ice_cream_parlor 185
/i/ice_floe 186
/i/ice_shelf 187
/i/ice_skating_rink/indoor 188
/i/ice_skating_rink/outdoor 189
/i/iceberg 190
/i/igloo 191
/i/industrial_area 192
/i/inn/outdoor 193
/i/islet 194
/j/jacuzzi/indoor 195
/j/jail_cell 196
/j/japanese_garden 197
/j/jewelry_shop 198
/j/junkyard 199
/k/kasbah 200
/k/kennel/outdoor 201
/k/kindergarden_classroom 202
/k/kitchen 203
/l/lagoon 204
/l/lake/natural 205
/l/landfill 206
/l/landing_deck 207
/l/laundromat 208
/l/lawn 209
/l/lecture_room 210
/l/legislative_chamber 211
/l/library/indoor 212
/l/library/outdoor 213
/l/lighthouse 214
/l/living_room 215
/l/loading_dock 216
/l/lobby 217
/l/lock_chamber 218
/l/locker_room 219
/m/mansion 220
/m/manufactured_home 221
/m/market/indoor 222
/m/market/outdoor 223
/m/marsh 224
/m/martial_arts_gym 225
/m/mausoleum 226
/m/medina 227
/m/mezzanine 228
/m/moat/water 229
/m/mosque/outdoor 230
/m/motel 231
/m/mountain 232
/m/mountain_path 233
/m/mountain_snowy 234
/m/movie_theater/indoor 235
/m/museum/indoor 236
/m/museum/outdoor 237
/m/music_studio 238
/n/natural_history_museum 239
/n/nursery 240
/n/nursing_home 241
/o/oast_house 242
/o/ocean 243
/o/office 244
/o/office_building 245
/o/office_cubicles 246
/o/oilrig 247
/o/operating_room 248
/o/orchard 249
/o/orchestra_pit 250
/p/pagoda 251
/p/palace 252
/p/pantry 253
/p/park 254
/p/parking_garage/indoor 255
/p/parking_garage/outdoor 256
/p/parking_lot 257
/p/pasture 258
/p/patio 259
/p/pavilion 260
/p/pet_shop 261
/p/pharmacy 262
/p/phone_booth 263
/p/physics_laboratory 264
/p/picnic_area 265
/p/pier 266
/p/pizzeria 267
/p/playground 268
/p/playroom 269
/p/plaza 270
/p/pond 271
/p/porch 272
/p/promenade 273
/p/pub/indoor 274
/r/racecourse 275
/r/raceway 276
/r/raft 277
/r/railroad_track 278
/r/rainforest 279
/r/reception 280
/r/recreation_room 281
/r/repair_shop 282
/r/residential_neighborhood 283
/r/restaurant 284
/r/restaurant_kitchen 285
/r/restaurant_patio 286
/r/rice_paddy 287
/r/river 288
/r/rock_arch 289
/r/roof_garden 290
/r/rope_bridge 291
/r/ruin 292
/r/runway 293
/s/sandbox 294
/s/sauna 295
/s/schoolhouse 296
/s/science_museum 297
/s/server_room 298
/s/shed 299
/s/shoe_shop 300
/s/shopfront 301
/s/shopping_mall/indoor 302
/s/shower 303
/s/ski_resort 304
/s/ski_slope 305
/s/sky 306
/s/skyscraper 307
/s/slum 308
/s/snowfield 309
/s/soccer_field 310
/s/stable 311
/s/stadium/baseball 312
/s/stadium/football 313
/s/stadium/soccer 314
/s/stage/indoor 315
/s/stage/outdoor 316
/s/staircase 317
/s/storage_room 318
/s/street 319
/s/subway_station/platform 320
/s/supermarket 321
/s/sushi_bar 322
/s/swamp 323
/s/swimming_hole 324
/s/swimming_pool/indoor 325
/s/swimming_pool/outdoor 326
/s/synagogue/outdoor 327
/t/television_room 328
/t/television_studio 329
/t/temple/asia 330
/t/throne_room 331
/t/ticket_booth 332
/t/topiary_garden 333
/t/tower 334
/t/toyshop 335
/t/train_interior 336
/t/train_station/platform 337
/t/tree_farm 338
/t/tree_house 339
/t/trench 340
/t/tundra 341
/u/underwater/ocean_deep 342
/u/utility_room 343
/v/valley 344
/v/vegetable_garden 345
/v/veterinarians_office 346
/v/viaduct 347
/v/village 348
/v/vineyard 349
/v/volcano 350
/v/volleyball_court/outdoor 351
/w/waiting_room 352
/w/water_park 353
/w/water_tower 354
/w/waterfall 355
/w/watering_hole 356
/w/wave 357
/w/wet_bar 358
/w/wheat_field 359
/w/wind_farm 360
/w/windmill 361
/y/yard 362
/y/youth_hostel 363
/z/zen_garden 364
cheers
Chris Maunder
|
|
|
|
|