您可能想知道如何检测网络摄像头视频Feed中的可疑行为？我们将使用您计算机的网络摄像头作为视频源，用于训练数据和测试您的神经网络模型。这种方法是使用迁移学习的监督学习。

你需要遵循什么

您应该可以访问安装了以下组件的计算机。

• Python 3
• Keras/Tensorflow
• Pillow （PIL）
• NumPy
• CV2

他们都可以通过pip和conda。

虽然，我已经在Mac上测试了这段Python代码，但它应该适用于任何系统。给出的文字转语音是唯一的例外，我以前用它subprocess.call()来调用Mac OS X say命令。您的操作系统上可能有一个等效的命令。

导入Python库

# Create training videos
import cv2
import numpy as np
from time import sleep
import glob
import os
import sys
from PIL import Image
import subprocess
NUM_FRAMES = 100
TAKES_PER = 2
CLASSES = ['SAFE', 'DANGER']
NEG_IDX = 0
POS_IDX = 1
HIDDEN_SIZE = 256
MODEL_PATH='model.h5'
TRAIN_MODEL = True
EPOCHS = 10
HIDDEN_SIZE = 16

准备数据

首先，我们需要一些训练数据来学习。我们需要“可疑”和“安全”行为的视频，因此请准备好行动！为了更容易训练我们的模型，您可以抓住玩具枪或其他可识别的物品来处理“可疑”场景。这样，在没有大量训练数据的情况下，您的模型将更容易分离两个案例。

这是一段Python代码片段，可从计算机的网络摄像头中捕获四个视频（两个可疑和两个安全），并将它们存储在一个data目录中供以后处理。

def capture(num_frames, path='out.avi'):
 
 # Create a VideoCapture object
 cap = cv2.VideoCapture(0)
 # Check if camera opened successfully
 if (cap.isOpened() == False): 
 print("Unable to read camera feed")
 # Default resolutions of the frame are obtained.The default resolutions are system dependent.
 # We convert the resolutions from float to integer.
 frame_width = int(cap.get(3))
 frame_height = int(cap.get(4))
 # Define the codec and create VideoWriter object.The output is stored in 'outpy.avi' file.
 out = cv2.VideoWriter(path, cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
 print('Recording started')
 for i in range(num_frames):
 ret, frame = cap.read()
 if ret == True: 
 # Write the frame into the file 'output.avi'
 out.write(frame)
 # When everything done, release the video capture and video write objects
 cap.release()
 out.release()
 
for take in range(VIDEOS_PER_CLASS):
 for cla in CLASSES:
 path = 'data/{}{}.avi'.format(cla, take)
 print('Get ready to act:', cla)
 # Only works on Mac
 subprocess.call(['say', 'get ready to act {}'.format(cla)])
 capture(FRAMES_PER_VIDEO, path=path)

看看data目录中的视频。你视频根据类别命名，例如SAFE1.avi用于安全视频。

使用预训练的模型从视频中提取特征

接下来，您需要将这些视频转换为机器学习算法可以训练的内容。为此，我们将重新利用经过预训练的VGG16网络，该神经网络已在ImageNet上接受过训练。Python实现如下：

# Create X, y series
from keras.preprocessing import image
from keras.applications.vgg16 import VGG16
from keras.applications.vgg16 import preprocess_input
import numpy as np
class VGGFramePreprocessor():
 
 def __init__(self, vgg_model):
 self.vgg_model = vgg_model
 
 def process(self, frame):
 img_data = cv2.resize(frame,(224,224))
 img_data = np.expand_dims(img_data, axis=0)
 img_data = preprocess_input(img_data)
 x = self.vgg_model.predict(img_data).flatten()
 x = np.expand_dims(x, axis=0)
 return x
def get_video_frames(video_path):
 vidcap = cv2.VideoCapture(video_path)
 success, frame = vidcap.read()
 while success:
 yield frame
 success,frame = vidcap.read()
 vidcap.release()
frame_preprocessor = VGGFramePreprocessor(VGG16(weights='imagenet', include_top=False))
 
if TRAIN_MODEL:
 # Load movies and transform frames to features
 movies = []
 X = []
 y = []
 for video_path in glob.glob('data/*.avi'):
 print('preprocessing', video_path)
 positive = CLASSES[POS_IDX] in video_path
 _X = np.concatenate([frame_preprocessor.process(frame) for frame in get_video_frames(video_path)])
 _y = np.array(_X.shape[0] * [[int(not positive), int(positive)]])
 X.append(_X)
 y.append(_y)
 X = np.concatenate(X)
 y = np.concatenate(y)
 print(X.shape)
 print(y.shape)

训练分类器

现在我们有了X和Y序列，现在是时候训练神经网络模型来区分可疑行为和安全行为了！在此示例中，我们将使用深度神经网络。你可以根据需要进行调整。Python代码如下：

from keras.models import Sequential, load_model
from keras.layers import Dense, Activation, Dropout
from sklearn.model_selection import train_test_split
from sklearn.metrics import f1_score
MODEL_PATH='model.h5'
EPOCHS = 10
HIDDEN_SIZE = 16
if TRAIN_MODEL:
 model = Sequential()
 model.add(Dense(HIDDEN_SIZE, input_shape=(X.shape[1],)))
 model.add(Dense(HIDDEN_SIZE))
 model.add(Dropout(0.2))
 model.add(Dense(len(CLASSES), activation='softmax'))
 model.compile(loss='categorical_crossentropy',
 optimizer='rmsprop',
 metrics=['accuracy'])
 x_train, x_test, y_train, y_test = train_test_split(X, y, random_state=42)
 model.fit(x_train, y_train,
 batch_size=10, epochs=EPOCHS,
 validation_split=0.1)
 model.save(MODEL_PATH)
 y_true = [np.argmax(y) for y in y_test]
 y_pred = [np.argmax(pred) for pred in model.predict(x_test)]
 score = f1_score(y_true, y_pred)
 print('F1:', score) 
else:
 model = load_model(MODEL_PATH)

准备测试！

现在到了有趣的部分。现在我们将使用我们构建的所有部分。是时候将计算机的网络摄像头变成现场CCTV行为检测器了！

# Infer on live video
from math import ceil
import subprocess
TEST_FRAMES = 500
# Initialize camera
cap = cv2.VideoCapture(0)
# Check if camera opened successfully
if (cap.isOpened() == False): 
 print("Unable to read camera feed")
 test_frames = 0
# Start processing video
for i in range(TEST_FRAMES):
 ret, frame = cap.read()
 if not ret: continue
 x_pred = frame_preprocessor.process(frame)
 y_pred = model.predict(x_pred)[0]
 conf_negative = y_pred[NEG_IDX]
 conf_positive = y_pred[POS_IDX]
 cla = CLASSES[np.argmax(y_pred)]
 if cla == CLASSES[POS_IDX]:
 subprocess.call(['say', CLASSES[POS_IDX]])
 progress = int(100 * (i / TEST_FRAMES))
 message = 'testing {}% conf_neg = {:.02f} conf_pos = {:.02f} class = {} 
'.format(progress, conf_negative, conf_positive, cla)
 sys.stdout.write(message)
 sys.stdout.flush()
cap.release()

结论

我希望你喜欢这个关于检测CCTV视频中可疑行为的教程。

一个明显的选择是在单一帧或帧序列上训练。为了简单起见，我为这个示例选择了单个帧，因为我们可以跳过一些正交任务，例如缓冲图像和排序训练数据。如果你想训练序列，你可以使用LSTM。