Click here to Skip to main content
15,867,488 members
Articles / Internet of Things / Arduino

Getting video stream from USB web-camera on Arduino Due - Part 1: Getting Started

Rate me:
Please Sign up or sign in to vote.
4.80/5 (21 votes)
8 Jun 2015CPOL10 min read 83.8K   786   41   18
Getting Started with Arduino Due in Atmel Studio

Introduction

The purpose of this project is to show how to get video stream from USB camera. However, one can also learn how to work with various parts of ARM Cortext-M3 SAM3X device, understand something about USB 2.0 and USB Video Class (UVC). I will explain what each piece of code does and what the correct sequences for various tasks are.

As you read those articles please note that this is my first ever embedded C project, my first ever Arduino Due project, my first ever USB and UVC project and my language of origin is not English.

Having already completed the project (it took me about half a year) I can now tell that Arduino Due is not enough to handle video, I suspected that at the beginning but I said myself I should try and then I would go from there. I was able to get video stream from web-camera and show it on TFT monitor in 2 resolutions: 160x120 and 176x144 in grayscale real-time without frame skipping. Maybe one of you will advance and get needed 320x240 (max for my TFT monitor), it is probably possible if using DMA controller and proper connection of TFT monitor. But any way I think you won't be able to get colour video as even uncompressed video stream requires lots of recalculations.

Let's start from listing the project's parts.

Project hardware

All parts togetherBesides personal computer following hardware is used:

  • Arduino Due;
  • 320x240 TFT monitor shield, take a note that this particular monitor is very dull and its surface like a mirror - a very bad monitor as for me or perhaps I don't know how to initialize it correctly. It would be very nice if someone could give more input about such TFT monitors;
  • Micro A plug – A receptacle USB cable;
  • A phone (not iPhone) USB data/charging A plug – micro B plug cable, everybody should have one of those somewhere;
  • A web-camera. I bought this one as it was the cheapest one on that online store.

Project software

I used free tools or free versions only:

  • Atmel Studio 6.x to write my code. It is a free package from Atmel. Looks like Visual Studios from Microsoft. Will require to fill a form before downloading;
  • Docklight a RS-232 monitor program to read debug messages from Arduino Due. I use free version that can't do some things which are not important for me in this project. Instead of Docklight one can use any other similar program.
  • Arduino software. We will not use Arduino software, it just installs USB driver for communication with the board and has a tool called bossac.exe. This tool is very important as the board is programmed with its help. I will not discuss details how to install it in Atmel Studio because people already described it nicely for example in this post.

Documents

I will refer to following documents in my text:

  • Definitive guide to ARM Cortex M3 2nd edition. This one gives you full understanding about Cortex-M3 and some differences to other versions. Not all sections of this big book is necessary to read. It can be legally bought or free downloaded from torrent trackers such as Pirate Bay.
  • SAM3X full specification. SAM3X8E is the processor on Arduino Due board. [1]
  • USB 2.0 specification. There is file usb_20.pdf inside the archive – very important document. [2]
  • USB Video Class 1.0 specification. I could not find it on their web-site because there are newer versions of this specification (1.1 and 1.5) available but it appeared that the camera I bought adheres to version 1.0 thus I downloaded the document from some Linux forum. [3]
  • USB Video Payload Uncompressed 1.0 specification. This document describes transport stream that is used by my camera. Your camera can have different transport stream and there is a document for each such stream. I will show later in the text how to find out your camera transport stream. [4]

Getting started

Image 2

Installation

Please install Atmel studio 6.x, Docklight, Arduino software. Connect Arduino Due to computer USB port, wait until OS installs USB drivers (provided in Arduino software folder), in driver's properties check which COM port number it uses for Arduino Due and remember it. Then read carefully above mentioned post about how to install external tool for Arduino Due programming. At the end it should look like mine on the picture. Note that my COM port number is 4, yours can be different.

To use it, simply compile the program with no errors, then hover your mouse over Tools menu and select Arduino Due programmer. Studio will upload your program to Arduino Due using bossac.exe utility.

Please take a note that later we'll use Docklight and because COM port cannot be shared it must be closed in Docklight before programming. Also every time you open COM port in Doclight restarts Arduino Due which is very useful feature.

After everything is installed let's start from embedded version of “Hello World” – blinking LED. This will get us started, teach us how to initialize main components of the microcontroller and give us an indication (blinking) that our code is actually working and is not hung up. A useful thing to do because I don’t have debugging device and Atmel Studio does not provide simulator for ARM processors.

Creating new project

Image 3Open Atmel Studio, select menu File -> New -> Project, select GCC C Executable Project, call it as you like. I'll refer to file with main function as main file further in the text.

In the next window select the target device. Arduino Due has ATSAM3X8E microcontroller, find it in the list, select and click OK.

During this several click process Atmel Studio did some job for us. Let's see.

In /cmsis/src forlder thare are 2 files: startup_sam3xa.c and system_sam3xa.c.

The code in the first file is executed even before our code: it defines vector table, initializes global variables to zero, C library, etc and there is a branch to the main function (that is where our program starts). I think this startup code is considered as advanced topic which I don't know much myself and I will not use it other than looking the correct name for an interrupt (exception). If you want to understand what it does, please read some articles about "bare-metal programming" like this.

In the following sections I will show what needs to be done to initialize Arduino Due, you will see all necessary registers and code sequences. I will also simplify the code created by Atmel Studio and put correct comments.

Basic Initialization

Second automatically created file does some basic initialization. Our main function in main file immediately calls SystemInit() that is located in system_sam3xa.c.

C++
#include "sam.h"

int main(void)
{
    SystemInit();
    while(1);
}

"Goto Implementation" of that function. First it initializes flash memory read operations, as Flash runs much slower than the core we need to specify wait states [1, p.1434]. From table 45-62 for 84MHz frequency wait state must be 4. It is exactly what is in SystemInit(). Wait states also mean that if a code has conditions and jumps, it will lower run speed as after each such operation the core will have to empty conveyer and wait 4 cycles (in our case) to access next instruction from the memory. Once it accessed the memory, it can fetch 2 or 4 instructions at a time so it does not need to wait again.

C++
EFC0->EEFC_FMR = EEFC_FMR_FWS(4);
EFC1->EEFC_FMR = EEFC_FMR_FWS(4);

After startup the microcontroller runs on embedded fast RC oscillator at 4MHz [1, p.522]. We need to switch it to run from much more stable 3-20MHz crystal oscillator which has attached 12MHz crystal according to Arduino Due schematic. That is what next piece of code does:

C++
if ( !(PMC->CKGR_MOR & CKGR_MOR_MOSCSEL) )
{
    PMC->CKGR_MOR = CKGR_MOR_KEY_PASSWD | SYS_BOARD_OSCOUNT | CKGR_MOR_MOSCRCEN | CKGR_MOR_MOSCXTEN;
    while ( !(PMC- >PMC_SR & PMC_SR_MOSCXTS) )
    {
    }
}

First it checks if 3-20MHz oscillator is not selected, we know that it is not. Then if not selected it enables it. And finally it waits until crystal oscillator comes up and stabilizes. I will drop what we don't need, code now looks like this:

C++
/* Initialize main oscillator */
PMC->CKGR_MOR |= CKGR_MOR_KEY_PASSWD | SYS_BOARD_OSCOUNT | CKGR_MOR_MOSCXTEN;
while ( !(PMC->PMC_SR & PMC_SR_MOSCXTS) );

Note that writing to Main oscillator register CKGR_MOR requires password which is value 0x37.

Once 3-20MHz crystal oscillator has been initialized, we switch to it from Embedded RC oscillator and wait until takeover is done:

PMC->CKGR_MOR = CKGR_MOR_KEY_PASSWD | SYS_BOARD_OSCOUNT | CKGR_MOR_MOSCRCEN | CKGR_MOR_MOSCXTEN | CKGR_MOR_MOSCSEL;
while ( !(PMC->PMC_SR & PMC_SR_MOSCSELS) );

Again, I simplify it by using OR not to show other parameters that we already used and to emphasise what exactly this step does:

C++
/* Switch to 3-20MHz Xtal oscillator */
PMC->CKGR_MOR |= CKGR_MOR_KEY_PASSWD | CKGR_MOR_MOSCSEL;
while ( !(PMC->PMC_SR & PMC_SR_MOSCSELS) );

Next piece of code switches to main clock which we don't need completely as after startup system runs from main clock [1, p.522]. Main clock comes from either embedded fast RC oscillator (which is default on startup) or 3-20MHz crystal oscillator which we initialized and selected just above. So we do not need to switch to main clock as it was already used from the beginning and following piece of code can be omitted:

C++
PMC->PMC_MCKR = (PMC->PMC_MCKR & ~(uint32_t)PMC_MCKR_CSS_Msk) | PMC_MCKR_CSS_MAIN_CLK;
while (!(PMC->PMC_SR & PMC_SR_MCKRDY));

What we need is to somehow change clock frequency from 12MHz to 84MHz. For that there is Divider and PLL (phase lock loop) [1, p.524]. Next piece of code initializes them:

C++
PMC->CKGR_PLLAR = SYS_BOARD_PLLAR;
while ( !(PMC->PMC_SR & PMC_SR_LOCKA) );

Please note that SYS_BOARD_PLLAR is defined on top of this file and supplies values described on page 524 section 27.6.1 of [1] and gives 168MHz output. It needs to be divided by 2 to obtain 84MHz, that is what next piece of code does:

C++
/* Switch to main clock */
PMC->PMC_MCKR = (SYS_BOARD_MCKR & ~PMC_MCKR_CSS_Msk) | PMC_MCKR_CSS_MAIN_CLK;
while ( !(PMC->PMC_SR & PMC_SR_MCKRDY) )
{
}

As you can see the comment is wrong and code is a bit complicated for this small operation. I will rewrite it as following:

C++
/* Setting up prescaler */
PMC->PMC_MCKR = PMC_MCKR_PRES_CLK_2 | PMC_MCKR_CSS_MAIN_CLK;
while ( !(PMC->PMC_SR & PMC_SR_MCKRDY) ) ;

It specifies prescaller (division by 2) and preserves Main clock selection. At this point everything is ready to switch to PLLA (84MHz) clock:

C++
/* Switch to PLLA */
PMC->PMC_MCKR = SYS_BOARD_MCKR;
while ( !(PMC->PMC_SR & PMC_SR_MCKRDY) )
{
}

Note that I removed SYS_BOARD_MCKR macros and used what it consists of. system_sam3xa.c looks now like this:

C++
#include "sam3xa.h"

#ifdef __cplusplus
extern "C" {
    #endif

    /* Clock settings (84MHz) */
    #define SYS_BOARD_OSCOUNT   (CKGR_MOR_MOSCXTST(0x8))
    #define SYS_BOARD_PLLAR     (CKGR_PLLAR_ONE | CKGR_PLLAR_MULA(0xdUL) 
                                 | CKGR_PLLAR_PLLACOUNT(0x3fUL) | CKGR_PLLAR_DIVA(0x1UL))

    void SystemInit( void )
    {
        /* Set FWS according to SYS_BOARD_MCKR configuration */
        EFC0->EEFC_FMR = EEFC_FMR_FWS(4);
        EFC1->EEFC_FMR = EEFC_FMR_FWS(4);

        /* Initialize main oscillator */
        PMC->CKGR_MOR |= CKGR_MOR_KEY_PASSWD | SYS_BOARD_OSCOUNT | CKGR_MOR_MOSCXTEN;
        while ( !(PMC->PMC_SR & PMC_SR_MOSCXTS) );

        /* Switch to 3-20MHz Xtal oscillator */
        PMC->CKGR_MOR |= CKGR_MOR_KEY_PASSWD | CKGR_MOR_MOSCSEL;
        while ( !(PMC->PMC_SR & PMC_SR_MOSCSELS) );

        /* Initialize PLLA */
        PMC->CKGR_PLLAR = SYS_BOARD_PLLAR;
        while ( !(PMC->PMC_SR & PMC_SR_LOCKA) );
        
        /* Setting up prescaler */
        PMC->PMC_MCKR = PMC_MCKR_PRES_CLK_2 | PMC_MCKR_CSS_MAIN_CLK;
        while ( !(PMC->PMC_SR & PMC_SR_MCKRDY) ) ;

        /* Switch to PLLA */
        PMC->PMC_MCKR = PMC_MCKR_PRES_CLK_2 | PMC_MCKR_CSS_PLLA_CLK;    
        while ( !(PMC->PMC_SR & PMC_SR_MCKRDY) ) ;
    }

    #ifdef __cplusplus
}
#endif

I/O initialization

To blink LEDs certain pins have to be configured as outputs, their voltage levels must change periodically.

There are several registers that control pins behaviour and they are grouped into PIO Controllers: PIOA, PIOB, etc. [1, p.618]. For them to operate properly we must enable their clocks [1, p.528 - 28.7]. I will enable all four PIO clocks although for now we need only PIOC and PIOA clock. TFT monitor connection will require usage of others PIOs too.

MC++
/* Activates clock to PIO controllers */
PMC->PMC_PCER0 =(1u << ID_PIOA) | (1u << ID_PIOB) | (1u << ID_PIOC) | (1u << ID_PIOD);

According to Arduino Due schematic, LED RX connected to PIOC line 30 and LED TX connected to PIOA line 21. See the following code for pins initialization, it is pretty self-explanatory and shows all necessary registers:

C++
/* RX LED pin initialization (output, default high)*/
uint32_t Pin = 1u << (PIO_PC30_IDX & 0x1F);    //Moving 1 to position 30
PIOC->PIO_IDR = Pin;                           //No interrupt
PIOC->PIO_PUDR = Pin;                          //No pull-up
PIOC->PIO_MDDR = Pin;                          //No open collector
PIOC->PIO_CODR = Pin;                          //Low level
PIOC->PIO_OER = Pin;                           //Output
PIOC->PIO_PER = Pin;                           //Enables
        
/* TX Pin pin initialization (output, default high) */
Pin = 1u << (PIO_PA21_IDX & 0x1F);
PIOA->PIO_IDR = Pin;
PIOA->PIO_PUDR = Pin;
PIOA->PIO_MDDR = Pin;
PIOA->PIO_CODR = Pin;
PIOA->PIO_OER = Pin;
PIOA->PIO_PER = Pin;

I added this code into function SystemInit() to the bottom. All initialization code will go here.

Timer initialization

Having pins configured is not enough to blink LEDs, some kind of periodical event is needed in which LEDs are switched on and off. Timers can raise such even. The simplest timer to use is SysTick [1, p.192]. Usually an operation system uses it to switch between tasks.

C++
/* Initialization of SysTick */
/* Reload value is every 1ms. -1 is for 1 tick to reload value */
SysTick->LOAD = ((84000000/1000) & SysTick_LOAD_RELOAD_Msk) - 1;
/* Clearing current value     */    
SysTick->VAL = 0;                                                        
/* Source is not divided by 8, enables SysTick, enables interrupt */
SysTick->CTRL = SysTick_CTRL_CLKSOURCE_Msk | SysTick_CTRL_ENABLE_Msk | SysTick_CTRL_TICKINT_Msk;   

Again, I added this code into function SystemInit() to bottom after pins initialization.

As we can see, SysTick interrupt is enabled and will be fired every 1ms. That means it must be caught and served.

Catching SysTick interrupt

Now as pins are configured and there is an event that fires every 1ms, it is time to make LEDs blink. We won't be able to see blinking with 1kHz frequency, but 1Hz is fine. Thus we need a variable which will be increased by 1 every interrupt. Once it reaches 1000 - we toggle pins and zero the variable. Move to our main file, add the variable and interrupt handler. To know the name of interrupt handler I refer to startup_sam3xa.c file:

C++
#include "sam.h"

uint32_t SysTickCounter;

void SysTick_Handler(void)
{    
    uint32_t Pin_LEDRX;
    uint32_t Pin_LEDTX;
    
    SysTickCounter ++;
    if(1000 == SysTickCounter)
    {
        SysTickCounter = 0;
        
        Pin_LEDRX = 1 << (PIO_PC30_IDX & 0x1Fu);
        Pin_LEDTX = 1 << (PIO_PA21_IDX & 0x1Fu);
        if(PIOC->PIO_PDSR & Pin_LEDRX)
        {
            PIOC->PIO_CODR = Pin_LEDRX;    //Lights on                                
            PIOA->PIO_CODR = Pin_LEDTX;
        }
        else
        {
            PIOC->PIO_SODR = Pin_LEDRX;    //Lights off                                
            PIOA->PIO_SODR = Pin_LEDTX;
        }
    }
}

int main(void)
{
    SystemInit();
    while(1);
}

Compile it, upload to Arduino Due.

It should look like this: http://youtu.be/pRXwDmMAFhE

Conclusion

In this article I shown some basics to start experimenting with Arduino Due not within the Arduino framework. In the next one I'll show how to initialize UART and how to print some messages back to your computer. I will create various print function for strings, number in hex, bin and decimal formats. Also I will initialize TFT monitor and show how to display something on it.

Source code is here.

Part two is here.

UPDATE 06-06-2015: Reloaded UVC 1.0 and UVC 1.0 uncompressed format specifications in zip (as pdfs are not allowed), updated links to USB 2.0 and SAM3X8E specifications as they have been moved to different locations. Page numbers in references to SAM3X8E reflect new edition of its specification.

UPDATE 10-07-2015: Reloaded source code without Debug folder.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer
Ukraine Ukraine
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
QuestionMerry Christmas and happy new year! Pin
Member 1409543019-Dec-18 19:24
Member 1409543019-Dec-18 19:24 
QuestionHow to use still image descriptor to take a picture Pin
baykouchev14-Oct-15 1:42
baykouchev14-Oct-15 1:42 
AnswerRe: How to use still image descriptor to take a picture Pin
Sergiy Bogdancev14-Oct-15 3:18
Sergiy Bogdancev14-Oct-15 3:18 
GeneralMy vote of 5 Pin
baykouchev13-Oct-15 1:03
baykouchev13-Oct-15 1:03 
Great article and very experienced author. Thank you so much!
GeneralRe: My vote of 5 Pin
Sergiy Bogdancev13-Oct-15 1:22
Sergiy Bogdancev13-Oct-15 1:22 
GeneralRe: My vote of 5 Pin
baykouchev14-Oct-15 1:39
baykouchev14-Oct-15 1:39 
QuestionWhy my cam Returns headers only Pin
baykouchev27-Sep-15 20:42
baykouchev27-Sep-15 20:42 
AnswerRe: Why my cam Returns headers only Pin
Sergiy Bogdancev27-Sep-15 20:57
Sergiy Bogdancev27-Sep-15 20:57 
AnswerRe: Why my cam Returns headers only Pin
Sergiy Bogdancev28-Sep-15 9:25
Sergiy Bogdancev28-Sep-15 9:25 
GeneralRe: Why my cam Returns headers only Pin
baykouchev28-Sep-15 10:55
baykouchev28-Sep-15 10:55 
AnswerRe: Why my cam Returns headers only Pin
Sergiy Bogdancev28-Sep-15 11:47
Sergiy Bogdancev28-Sep-15 11:47 
AnswerRe: Why my cam Returns headers only Pin
Sergiy Bogdancev28-Sep-15 21:42
Sergiy Bogdancev28-Sep-15 21:42 
GeneralRe: Why my cam Returns headers only Pin
baykouchev29-Sep-15 6:09
baykouchev29-Sep-15 6:09 
GeneralRe: Why my cam Returns headers only Pin
Sergiy Bogdancev29-Sep-15 6:45
Sergiy Bogdancev29-Sep-15 6:45 
GeneralRe: Why my cam Returns headers only Pin
Sergiy Bogdancev29-Sep-15 21:12
Sergiy Bogdancev29-Sep-15 21:12 
QuestionHello world blink Pin
Phebous13-Jul-15 16:30
Phebous13-Jul-15 16:30 
Questionvery nice Pin
BillW3317-Mar-15 5:19
professionalBillW3317-Mar-15 5:19 
QuestionNice work Pin
Mike Hankey12-Mar-15 18:38
mveMike Hankey12-Mar-15 18:38 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.