Decoding video packets received from tcpreplay using C++/qt

I have a PCAP file having the encoded video data. I am using TCPReplay to broadcast this PCAP data to my server.

In my server I have written a QT UDP socket program to receive these packets. I am able to receive these packets successfully in my server and stored the data in QByteArray.

Here I am using PCAP file and TCPReplay as a simulator. In reality these packets will be received from another process/server.

I would like to decode these packets. How can I decode these packets in C++/Qt ?

What data structure to use ? Do I need to use FFMPEG or gstreamer APIs ?

I am completely new to video decoding.

So please elaborate answer with some sample code

#include "streamingserver.h"

StreamingServer::StreamingServer(QObject *parent) : QObject(parent), messageCount(0)
{
    socket = new QUdpSocket(this);

    connect(socket, SIGNAL(connected()), this, SLOT(connected()));
    connect(socket, SIGNAL(disconnected()), this, SLOT(disconnected()));
    connect(socket, SIGNAL(readyRead()), this, SLOT(readyRead()));

    if(!socket->bind(QHostAddress::Any, 50001))
    {
        qDebug() << "Server could not start!";
    }
    else
    {
        qDebug() << "Server started!";
    }
}

void StreamingServer::connected()
{
    qDebug() << "Connected";
}

void StreamingServer::disconnected()
{
    qDebug() << "Disconnected";
}

void StreamingServer::readyRead()
{
    // when data comes in
    QByteArray buffer;

    buffer.resize(socket->pendingDatagramSize());

    QHostAddress sender;
    quint16 senderPort;

    socket->readDatagram(buffer.data(), buffer.size(), &sender, &senderPort);

    qDebug() << "Message Count: " << ++messageCount;
    qDebug() << "Message From: " << sender.toString();
    qDebug() << "Message Port: " << senderPort;
    qDebug() << "Message Size: " << buffer.size();
    qDebug() << "Message: ";
    qDebug() << buffer;

    qDebug() << "******************************************************************************";

}