15:46:08:ObjectDetection (Coral): Retrieved objectdetection_queue command
15:46:10:objectdetection_coral_adapter.py: Output details: {'name': 'StatefulPartitionedCall:3;StatefulPartitionedCall:2;StatefulPartitionedCall:1;StatefulPartitionedCall:02', 'index': 8, 'shape': array([ 1, 20], dtype=int32), 'shape_signature': array([ 1, 20], dtype=int32), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Input details: {'name': 'serving_default_input:0', 'index': 0, 'shape': array([ 1, 300, 300, 3], dtype=int32), 'shape_signature': array([ 1, 300, 300, 3], dtype=int32), 'dtype': , 'quantization': (0.007843137718737125, 127), 'quantization_parameters': {'scales': array([0.00784314], dtype=float32), 'zero_points': array([127], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Output details: {'name': 'StatefulPartitionedCall:3;StatefulPartitionedCall:2;StatefulPartitionedCall:1;StatefulPartitionedCall:02', 'index': 8, 'shape': array([ 1, 20], dtype=int32), 'shape_signature': array([ 1, 20], dtype=int32), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Input details: {'name': 'serving_default_input:0', 'index': 0, 'shape': array([ 1, 300, 300, 3], dtype=int32), 'shape_signature': array([ 1, 300, 300, 3], dtype=int32), 'dtype': , 'quantization': (0.007843137718737125, 127), 'quantization_parameters': {'scales': array([0.00784314], dtype=float32), 'zero_points': array([127], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Output details: {'name': 'StatefulPartitionedCall:3;StatefulPartitionedCall:2;StatefulPartitionedCall:1;StatefulPartitionedCall:02', 'index': 8, 'shape': array([ 1, 20], dtype=int32), 'shape_signature': array([ 1, 20], dtype=int32), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Input details: {'name': 'serving_default_input:0', 'index': 0, 'shape': array([ 1, 300, 300, 3], dtype=int32), 'shape_signature': array([ 1, 300, 300, 3], dtype=int32), 'dtype': , 'quantization': (0.007843137718737125, 127), 'quantization_parameters': {'scales': array([0.00784314], dtype=float32), 'zero_points': array([127], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Output details: {'name': 'StatefulPartitionedCall:3;StatefulPartitionedCall:2;StatefulPartitionedCall:1;StatefulPartitionedCall:02', 'index': 8, 'shape': array([ 1, 20], dtype=int32), 'shape_signature': array([ 1, 20], dtype=int32), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Refreshing the Tensorflow Interpreter
15:46:10:objectdetection_coral_adapter.py: Refreshing the Tensorflow Interpreter
15:46:10:objectdetection_coral_adapter.py: Refreshing the Tensorflow Interpreter
15:46:10:objectdetection_coral_adapter.py: Input details: {'name': 'serving_default_input:0', 'index': 0, 'shape': array([ 1, 300, 300, 3], dtype=int32), 'shape_signature': array([ 1, 300, 300, 3], dtype=int32), 'dtype': , 'quantization': (0.007843137718737125, 127), 'quantization_parameters': {'scales': array([0.00784314], dtype=float32), 'zero_points': array([127], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Refreshing the Tensorflow Interpreter
15:46:10:objectdetection_coral_adapter.py: Output details: {'name': 'StatefulPartitionedCall:3;StatefulPartitionedCall:2;StatefulPartitionedCall:1;StatefulPartitionedCall:02', 'index': 8, 'shape': array([ 1, 20], dtype=int32), 'shape_signature': array([ 1, 20], dtype=int32), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Input details: {'name': 'serving_default_input:0', 'index': 0, 'shape': array([ 1, 300, 300, 3], dtype=int32), 'shape_signature': array([ 1, 300, 300, 3], dtype=int32), 'dtype': , 'quantization': (0.007843137718737125, 127), 'quantization_parameters': {'scales': array([0.00784314], dtype=float32), 'zero_points': array([127], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Output details: {'name': 'StatefulPartitionedCall:3;StatefulPartitionedCall:2;StatefulPartitionedCall:1;StatefulPartitionedCall:02', 'index': 8, 'shape': array([ 1, 20], dtype=int32), 'shape_signature': array([ 1, 20], dtype=int32), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Input details: {'name': 'serving_default_input:0', 'index': 0, 'shape': array([ 1, 300, 300, 3], dtype=int32), 'shape_signature': array([ 1, 300, 300, 3], dtype=int32), 'dtype': , 'quantization': (0.007843137718737125, 127), 'quantization_parameters': {'scales': array([0.00784314], dtype=float32), 'zero_points': array([127], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Output details: {'name': 'StatefulPartitionedCall:3;StatefulPartitionedCall:2;StatefulPartitionedCall:1;StatefulPartitionedCall:02', 'index': 8, 'shape': array([ 1, 20], dtype=int32), 'shape_signature': array([ 1, 20], dtype=int32), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Input details: {'name': 'serving_default_input:0', 'index': 0, 'shape': array([ 1, 300, 300, 3], dtype=int32), 'shape_signature': array([ 1, 300, 300, 3], dtype=int32), 'dtype': , 'quantization': (0.007843137718737125, 127), 'quantization_parameters': {'scales': array([0.00784314], dtype=float32), 'zero_points': array([127], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Refreshing the Tensorflow Interpreter
15:46:10:objectdetection_coral_adapter.py: Refreshing the Tensorflow Interpreter
15:46:10:objectdetection_coral_adapter.py: Refreshing the Tensorflow Interpreter
15:46:10:objectdetection_coral_adapter.py: Output details: {'name': 'StatefulPartitionedCall:3;StatefulPartitionedCall:2;StatefulPartitionedCall:1;StatefulPartitionedCall:02', 'index': 8, 'shape': array([ 1, 20], dtype=int32), 'shape_signature': array([ 1, 20], dtype=int32), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Input details: {'name': 'serving_default_input:0', 'index': 0, 'shape': array([ 1, 300, 300, 3], dtype=int32), 'shape_signature': array([ 1, 300, 300, 3], dtype=int32), 'dtype': , 'quantization': (0.007843137718737125, 127), 'quantization_parameters': {'scales': array([0.00784314], dtype=float32), 'zero_points': array([127], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Output details: {'name': 'StatefulPartitionedCall:3;StatefulPartitionedCall:2;StatefulPartitionedCall:1;StatefulPartitionedCall:02', 'index': 8, 'shape': array([ 1, 20], dtype=int32), 'shape_signature': array([ 1, 20], dtype=int32), 'dtype': , 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}
15:46:10:objectdetection_coral_adapter.py: Refreshing the Tensorflow Interpreter
15:46:10:objectdetection_coral_adapter.py: Refreshing the Tensorflow Interpreter
15:46:10:ObjectDetection (Coral): Rec'd request for ObjectDetection (Coral) command 'detect' (...b8118a) took 2689ms
After this blurp in the log the TPU is not working, The UI still says GPU (TPU) but the numbers are > 1sec, otherwise it takes ~20ms
Probably switching to CPU
17:11:32:ObjectDetection (Coral): Retrieved objectdetection_queue command
17:11:33:ObjectDetection (Coral): Rec'd request for ObjectDetection (Coral) command 'detect' (...b80a07) took 1003ms
17:11:52:ObjectDetection (Coral): Retrieved objectdetection_queue command
17:11:53:ObjectDetection (Coral): Rec'd request for ObjectDetection (Coral) command 'detect' (...2532f7) took 1006ms
17:12:04:ObjectDetection (Coral): Retrieved objectdetection_queue command
17:12:05:ObjectDetection (Coral): Rec'd request for ObjectDetection (Coral) command 'detect' (...020032) took 1003ms
17:12:35:ObjectDetection (Coral): Retrieved objectdetection_queue command
17:12:36:ObjectDetection (Coral): Rec'd request for ObjectDetection (Coral) command 'detect' (...58423f) took 1003ms
I can detect this by periodic tests from my monitoring system by sending a detection request with a known image periodically. So I can automate the restarts or CPAI can implement service autorestart?
|