Table of Contents
Introduction
Background
Running Gallery Server Pro
Solution Architecture
User Management and Security
Entity Framework Code First Migrations
A Template-Based User Interface
Rendering HTML for Video, Images, Audio and More
Transcoding video and audio with FFmpeg
Media Object, Albums and the Composite Pattern
Using the Strategy Pattern for Persisting to the Data Store
Summary
Gallery Server Pro is a powerful and easy-to-use Digital Asset Management (DAM) application and web gallery for sharing
and
managing photos, video, audio, and other files. It is open source software released under GPL v3 and written with
ASP.NET and C#. The entire application is contained in a single ASCX user control for easy insertion into your own web site.
- Organize your media files into albums you can easily add, edit, delete, rotate, sort, copy and move
- UI based on jsRender templates you can easily modify
- Uses HTML5 and CSS3 for plug-free video and audio, falling back to Flash and Silverlight when necessary
- Add files using one-click synchronize and ZIP file upload functions. Thumbnail and compressed versions are automatically
created
- Supports external media objects such as YouTube videos
- Scalable to hundreds of thousands of objects
- Powerful user security with flexible, per-album granularity
- Metadata extraction
- Image watermarking
- SQL CE or SQL Server database
- 100% managed code written in C# and ASP.NET 4.5
- Source code is released under the open source GNU General Public License v3
You can play with an
online demo of Gallery Server Pro to get a sense of its capabilities. A pre-compiled version is available, including additional documentation and a support forum.
This project started in 2002 from my desire to share my photos over the web. I wanted my photos to remain on my own server,
not somebody else's like Flicker or
Facebook. Since there weren't any free solutions to choose from at the time, I wrote my own.
Version 1 was released to the world in January 2006. Since that time, there have been hundreds of thousands of downloads
and a steady stream of releases. At the time of this writing, the latest version is
3.0.3. Check
galleryserverpro.com for the latest version.
In this article, I present the overall architecture and technical features of Gallery Server Pro. The topics presented here
can help if you want to learn more about:
- Implementing a web gallery to share photos, video, audio, and other documents
- Using Entity Framework Code First Migrations
- Rendering HTML with jsRender
- Creating an application that lets users modify jsRender templates from the site admin area
- Using FFmpeg to transcode video and audio from ASP.NET
- Using the composite design pattern to manage infinitely
hierarchical relationships.
- When and how to use the strategy design pattern
Gallery Server Pro is a fully functional and stable web application ready for production use. To use the compiled
version, visit the download page and follow
the installation instructions in the Admin Guide.
Since you're reading this article, you may prefer to start with the source code. You'll need to use Visual Studio
2012
or higher. Here's how:
- Download the source code. Unblock the ZIP
file before extracting the contents. (Right-click the file in Windows Explorer, choose Properties, and then click
'Unblock' on the General tab. If you don't see this button, then ignore this message.)
- Extract the directories and files to the desired location.
- Start Visual Studio and open the solution file.
- The code is configured to use a SQL CE database by default. If you want to use SQL Server, open web.config, uncomment
the SQL Server connection string and edit it to provide credentials. Then comment out or delete the SQL CE connection
string.
- That's it! Press F5 to run the website. The gallery should automatically appear in the browser. Click the link
to create an admin account. At this point you can use the gallery just as you would in a normal installation.
Gallery Server Pro stores media objects in a hierarchal set of albums. A media object is any type of file or a chunk of
HTML code like the embed code you find on many websites. Media files and albums are stored in a directory named gs\mediaobjects
within the web application. (This can be changed to any
network-accessible location.) An album is really just a directory, so an album named Vacation Photos is stored as a similarly
named directory.
There are two main techniques for adding media objects:
- Upload a ZIP file containing the media files. If the ZIP file contains directories, they are converted to albums.
- Copy your media files to the media objects directory, and then start a synchronization in Gallery Server Pro.
When adding a media object, the following steps occur:
- The file is saved to the media objects directory. (If adding a media object via the synchronization technique, then this
step is already done.)
- Metadata about the file (eg. camera model, shutter speed, video length, etc) is extracted.
- A thumbnail image is created and saved to the hard drive.
- A compressed, bandwidth-friendly version is created for images, videos, and audio files.
- A record is added to the data store to represent this media object.
Media objects are streamed to the browser through an HTTP handler. Below you can see a photo and a video being displayed.
If watermarking is enabled, the watermark is applied to the in-memory version of the image just before it is sent.
The right pane shows metadata about the media object. The layout and editability of these items is highly configurable.
By default, everyone can browse the media objects. However, you must log on to perform any action that modifies an album
or media object. Authorization to modify data is configured by type of permission and the albums to which it applies. For
example, you can set up user Soren to have edit permission to the album Soren's photos.
Another user Margaret is given edit permission to the album Margaret's photos. Each user
can administer his or her own album but cannot edit albums outside his or her domain.
To learn more about how to use Gallery Server Pro from an end-user perspective, read the
Administrator's Guide.
Otherwise, read on to learn about the architecture and programming techniques.
The source code contains six projects. They are:
Project name | Description |
Website | UI layer - ASP.NET 4.5 web application |
TIS.GSP.Business | Business layer logic |
TIS.GSP.Business.Resources | Contains resource data to support the business layer |
TIS.GSP.Business.Interfaces | Defines all interfaces used in the solution |
TIS.GSP.Events
| Provides event handling support |
TIS.GSP.Data.Data | Data layer logic |
User accounts are managed through the ASP.NET Membership and Roles APIs. By default, Gallery Server Pro is configured to
use a locally stored
SQL CE database named GalleryDb.sdf in the App_Data directory. It interacts with this database by using
System.Web.Providers.DefaultMembershipProvider
for users and System.Web.Providers.DefaultRoleProvider
for
roles.
Because of the flexibility offered by the provider model, you can use any data store that has a membership provider. For
example, you can use SqlMembershipProvider
to use SQL Server or ActiveDirectoryMembershipProvider
to plug Gallery Server Pro into your existing base of Active Directory users. The
Administrator's Guide
contains a section for Membership Configuration that provides more information.
Overview
Versions of Gallery Server Pro prior to 3.0 used the data provider model for interacting with the database. This
required creating classes that inherited from System.Configuration.Provider.ProviderBase
to provide specific
implementations for each supported data technology. This approach has some nice benefits, such as being able to target
multiple databases and allowing each data
implementation to take advantage of the strengths of the platform. For example, the SQL Server data provider used stored
procedures for all data access.
However, this approach has a serious drawback—all data access code must be written
once for each data provider. Since Gallery Server Pro shipped with two providers—SQL CE and SQL Server—that
meant duplicating every piece of data access code. Of course, that doubled the testing as well. To further complicate
things, the upgrade script that updated the data schema for each version had to be duplicated. This duplication of code
violates the DRY principle, slows down development and is,
frankly, not a lot of fun to develop and maintain.
So we switched to Entity Framework Code First Migrations for version 3.0. The primary benefit is that we write one
chunk of LINQ code to interact with the database. The EF framework takes care of generating the correct SQL
syntax for SQL CE or SQL Server. For example, to retrieve an album from the database we only need to write this:
using (var repo = new AlbumRepository())
{
var album = repo.Where(a => a.AlbumId == albumId);
}
EF handles the rest for us, meaning we don't have to mess with ADO.NET commands, connections, or database-specific
intricacies. Even better, migrations support means we can add a column to a table with database-agnostic code like this:
AddColumn("gsp.Metadata", "RawValue", c => c.String(maxLength: 1000));
When the gallery web application starts, EF initialization code checks the current version of the database,
automatically upgrading it if necessary.
Database requirements
We had these requirements EF Code First Migrations had to handle:
- Automatically create the database, objects and initial data the first time the gallery starts or any time it is
missing.
- If the database exists but doesn't yet have any gallery objects, create the objects and seed it with initial data.
- If an earlier version of the gallery data schema is present, automatically upgrade it to the current version. It must
support upgrades from any previous version of Gallery Server Pro going back to 3.0.0.
These requirements must be simultaneously met without requiring changes to web.config or any other user input. As it
turns out, it was a real challenge but we got there in the end.
Create and update the database
To have EF automatically manage the database, including creating, seeding, and updating as required, we run the following code during startup:
private static void InitializeDataStore()
{
System.Data.Entity.Database.SetInitializer(new System.Data.Entity.MigrateDatabaseToLatestVersion<GalleryDb, GalleryDbMigrationConfiguration>());
var configuration = new GalleryDbMigrationConfiguration();
var migrator = new System.Data.Entity.Migrations.DbMigrator(configuration);
if (migrator.GetPendingMigrations().Any())
{
migrator.Update();
}
}
This code registers the MigrateDatabaseToLatestVersion
database initializer and passes in a reference to the
GalleryDbMigrationConfiguration
class. Then it checks to see if there are pending migrations and, if so, executes them. The GalleryDbMigrationConfiguration
class looks like this:
public sealed class GalleryDbMigrationConfiguration : DbMigrationsConfiguration<GalleryDb>
{
protected override void Seed(GalleryDb ctx)
{
MigrateController.ApplyDbUpdates();
}
}
The Seed method is called once all migrations are complete, so we can be guaranteed the database schema matches the
Code First data model (assuming we've written the migration Up() methods correctly). This method is a good place to
insert seed data or modify data that is required to support the new version. For example, the AppSetting table contains
a record that stores
the current data schema version (eg. "3.0.3"). When the gallery is updated to a newer version, this record must
be
updated as well.
At first one might think a good place for updating a record is in the migration's Up() method, and in fact early
versions of Gallery Server Pro did this. However, this approach can fail when the schema is updated. For example, let's
say we add a new column to the AppSetting table. The Up() method for the migration will include a call to AddColumn to
add the new column, but the change won't actually propagate to the table until some point after exiting the Up()
method. That means any attempt to query the AppSetting table from the Up() method will fail because the entity model
(eg. the Code First AppSetting
class) includes the new column but the database table doesn't.
This is why the Seed method is a good place to insert, update, or delete database records. We wrote a MigrateController
class
to handle these data updates:
public static void ApplyDbUpdates()
{
using (var ctx = new GalleryDb())
{
if (!ctx.AppSettings.Any())
{
SeedController.InsertSeedData(ctx);
}
var curSchema = GetCurrentSchema(ctx);
while (curSchema < GalleryDb.DataSchemaVersion)
{
var oldSchema = curSchema;
switch (curSchema)
{
case GalleryDataSchemaVersion.V3_0_0: UpgradeTo301(); break;
case GalleryDataSchemaVersion.V3_0_1: UpgradeTo302(); break;
case GalleryDataSchemaVersion.V3_0_2: UpgradeTo303(); break;
case GalleryDataSchemaVersion.V3_0_3: UpgradeTo310(); break;
}
curSchema = GetCurrentSchema(ctx);
if (curSchema == oldSchema)
{
throw new Exception(String.Format("The migration function for schema {0} should have incremented the data schema version in the AppSetting table, but it did not.", curSchema));
}
}
}
}
private static GalleryDataSchemaVersion GetCurrentSchema(GalleryDb ctx)
{
return GalleryDataSchemaVersionEnumHelper.ConvertGalleryDataSchemaVersionToEnum(ctx.AppSettings.First(a => a.SettingName == "DataSchemaVersion").SettingValue);
}
The first thing the method does is insert seed data if no records are found in the AppSetting table. This will be the case
when installing the gallery for the first time. The InsertSeedData() method populates several tables with lookup data
and default settings. Refer to the source code if you're curious.
Next, the function gets the current data schema version from the AppSetting table in the database and uses it to call the
corresponding update
method. Refer to the source code to see these functions, but suffice it to say they update various tables like
MediaTemplate and UiTemplate to implement bug fixes and update behavior. They also increment the data schema version in
the AppData table.
We won't get into detail about how the migrations are created because that is pretty standard Code First Migrations
stuff that is easily
found on the internet. In short, we executed commands in the Package Manager Console of Visual Studio to enable
migrations (Enable-Migrations) and add them (eg. Add-Migration v3.1.0). It is not necessary to call Update-Database
because the the MigrateDatabaseToLatestVersion
database initializer handles that for us the first time we
run the application.
Specifying the database
You may have noticed the above node doesn't explicitly refer to SQL Server or SQL CE. Instead, it is database-agnostic
and works equally well against either one. Well then, where do we specify which database technology we want to use and
other relevant information such as the authentication info? This is all taken care of with the connection string
in the web.config file in the root of the web application:
<connectionStrings>
<clear />
<add name="GalleryDb" providerName="System.Data.SqlClient" connectionString="server=(local);uid=;pwd=;Trusted_Connection=yes;database=GalleryServerPro;Application Name=Gallery Server Pro;MultipleActiveResultSets=True" />
</connectionStrings>
The DbContext
in our application is named GalleryDb
, so EF looks for a connection string
with that name. In the example above, the connection string specifies the provider System.Data.SqlClient
(SQL Server) and tells EF where the
server is, how to authenticate, and the name of the database. When the application starts, EF looks for the database,
creating it if necessary. Then it creates the database objects and seeds it with data. By the time the first page loads,
you have a fully configured database and your gallery is ready to go.
If one wanted to use SQL CE instead, the only required change is an update to the connection string:
<connectionStrings>
<clear />
<add name="GalleryDb" providerName="System.Data.SqlServerCe.4.0" connectionString="data source=|DataDirectory|\GalleryData.sdf" />
</connectionStrings>
The next time the application starts, EF detects the updated connection string, creates
the SQL CE database, seeds it with data, and begins using it.
This behavior is really nice during development, because it means one can delete the database at any time and have it
auto-created during the next page load. Since this is the same process that occurs for end users when they install the
gallery, it is a convenient way to test the installation process to verify that changes in default settings or other
setup behavior is occurring as expected.
NOTE: If you place an empty text file named install.txt in the App_Data directory, the startup behavior changes slightly.
A link will appear on the main page that lets you create an administrator account. Since you need an admin account to
manage the gallery, this is typically something you'll want to do at the same time you have EF create the database.
Overview
UI templates are a powerful feature that lets you modify the user interface using only HTML and JavaScript skills. For example,
here are a few things you can do with UI templates:
- Add your company logo to the header
- Add an image preview when hovering over a thumbnail
- Add links in the album tree to common tags
- Show the five most recently added items at the top of an album
We'll look at how to do all these examples—and a few more—later in this section. But first let's get the basics down.
There are five sections in the gallery that are built from UI templates:
These images show the five templates: header, left pane, right pane, album, and media object. The center pane shows either
the album or media object template, depending on whether the user is looking at an album or an individual media object.
What is NOT built from UI templates?
The main browsing interface is built from templates, but it is important to understand what parts are not:
Actions menu and album breadcrumb – The Actions menu and the album breadcrumb links are not part of any
template. Instead, its HTML is generated on the server and inserted into the page under the header. A future version of
Gallery Server Pro may merge this portion into the header template so that the entire page (as seen in the screen shots
above) is 100% template driven.
Task pages – These are the pages that carry out a task, such as uploading files, editing captions, etc.
These pages are ASCX user controls and not based on a UI template.
Site admin area – None of the pages in the site admin area are built from UI templates. As with the task
pages, they are ASCX user controls and can be modified by editing the source code.
Each section shown above is rendered from a UI template. A UI template consists of a chunk of HTML containing jsRender syntax
and some JavaScript that tells the browser what to do with the HTML. Typically the script invokes the jsRender template
engine and appends the generated HTML to the page. A robust set of gallery data is available on the client that gives you
access to important information such as the album and media object data, user permissions, and gallery settings. We'll get
into the structure of this data later in this post.
You can view the UI template definitions on the UI Templates page in the site admin area. In these two images you can see
the HTML and JavaScript values:
You can assign which albums this particular template applies to on the Target Albums tab. If multiple templates target the
same album, the most specific one wins. For example, if the default template is assigned to 'All albums' and a second template
is assigned to the Samples album, the second template will be used for the Samples album and any of its children because
that template definition is 'closer' to the album.
The preview tab lets you see the result of any edits you make before saving the changes.
Anatomy of the left pane
Let's take a close look at how one of the templates works. We'll choose the left pane template first. The HTML is simple:
<div id='{{:Settings.ClientId}}_lptv'></<div>
It defines a single, empty div tag and gives it a unique ID. The text between the double brackets is jsRender syntax that
refers to the ClientId property of the Settings object. This particular property provides a string that is unique to the
current instance of the Gallery user control on the page. When it is rendered on the page you end up with HTML similar to
this:
<div id='gsp_g_lptv'></<div>
Note: Defining a unique ID is not required for most galleries, but it is there for admins who want to include two instances
of the gallery control on a page. For example, you might have a slideshow running in one part of a web page and a video
playing in another.
Astute observers will notice that a single div tag doesn't look anything like a complex treeview. So how does that div tag
eventually become the treeview? Let's look at the JavaScript that is part of the template:
var isTouch = window.Gsp.isTouchScreen();
var renderLeftPane = !isTouch || (isTouch && ($('.gsp_tb_s_CenterPane:visible, .gsp_tb_s_RightPane:visible').length == 0));
if (renderLeftPane ) {
$('#{{:Settings.LeftPaneClientId}}').html( $.render [ '{{:Settings.LeftPaneTmplName}}' ]( window.{{:Settings.ClientId}}.gspData ));
var options = {
albumIdsToSelect: [{{:Album.Id}}],
navigateUrl: '{{:App.CurrentPageUrl}}'
};
$('#{{:Settings.ClientId}}_lptv').gspTreeView(window.{{:Settings.ClientId}}.gspAlbumTreeData, options);
}
Technically this text is not pure JavaScript. See the jsRender syntax in there? That's right, it is ALSO a jsRender template
that will be run through the jsRender engine to produce pure JavaScript just before execution. This is an extraordinarily
powerful feature and can be harnessed to produce a wide variety of UI possibilities. Imagine writing script that loops through
the images in an album to calculate some value or invoke a callback to the server to request data about a specific album.
The first thing the script does is decide whether the left pane template should even be appended to the page. We decide
not
to show it on touchscreens for two reasons: (1) touchscreens typically have small screens and cannot afford the real estate
required by the left pane (2) the splitter control that separates the left, center, and right panes does not work well on
touchscreens.
The script refers to the function window.Gsp.isTouchScreen()
. You'll find this function in the minified file
gs\script\gallery.min.js.
If you are editing the gallery you will probably prefer to have un-minified script files loaded into the browser. You can
easily accomplish this by switching the debug setting to 'true' in web.config.
If the script decides the left pane is to be rendered, it executes this line:
$('#{{:Settings.LeftPaneClientId}}').html( $.render [ '{{:Settings.LeftPaneTmplName}}' ]( window.{{:Settings.ClientId}}.gspData ));
This will look familiar to anyone with jsRender experience or other types of template engines. In plain English, it says
to take the template having the name LeftPaneTmplName and the data in the variable gspData, run in through the jsRender
engine, and assign the resulting HTML to the HTML element named LeftPaneClientId. In other words, it takes the string
<div
id='{{:Settings.ClientId}}_lptv'></div>
, converts it to
<div id='gsp_g_lptv'></div>
,
and adds it to
the page's HTML DOM.
So far all the script has done is add a div tag to the page, but we still need to convert it into the album tree. That's
what the last section does:
var options = {
albumIdsToSelect: [{{:Album.Id}}],
navigateUrl: '{{:App.CurrentPageUrl}}'
};
$('#{{:Settings.ClientId}}_lptv').gspTreeView(window.{{:Settings.ClientId}}.gspAlbumTreeData, options);
The JavaScript file I mentioned above contains several jQuery plug-ins to help reduce the complexity of the templates and
to maximize the amount of script that is cached by the browser (inline script is never cached). Here we use jQuery to grab
a reference to our generated div tag and we invoke the gspTreeView plug-in on it, passing along the album treeview data
and a few options. The tree data is a JSON object containing the album structure and is included in every page request.
In turn, the gspTreeView plug-in is a wrapper around the third party jQuery tree control jsTree, the code for which is in
the lib.min.js file (or lib.js if you are running with debug=true). There are several third party script libraries in that
file.
The treeview plug-in builds an HTML tree from the data and appends it to the div tag, resulting in the tree view you see
in the left pane. You can play with the HTML and JavaScript and then use the preview tab to see how those edits affect the
output.
Client data model
Each page contains a rich set of data that is included with the browser request. A global JavaScript variable named gspData
exists that is scoped to the current gallery user control instance. In a default installation, you can find it at window.gsp_g.gspData,
as seen in this image from Chrome:
Let's take a brief look at each top-level property:
ActiveGalleryItems – An array of GalleryItem instances that are currently selected. A gallery item can
represent an album or media object (photo, video, audio, document, YouTube snippet, etc.)
ActiveMetaItems – An array of MetaItem instances describing the currently selected item(s).
Album – Information about the current album. It has two important properties worth explaining: GalleryItems
and MediaItems. Both represent albums and media objects, but a GalleryItem instance contains only basic information about
each item while a MediaItem instance contains all the metadata and other details about an item. Because a GalleryItem instance
is lightweight, it is well suited for album thumbnail views where you only need basic information. Therefore, to optimize
the data passed to the browser, the MediaItems property is null when viewing an album and the GalleryItems property is null
when viewing a single media object.
App – Information about application-wide settings, such as the skin path and current URL.
Resource – Contains language resources.
Settings – Contains gallery-specific settings.
User – Information about the current user.
Here is a complete list of client objects and their properties (click images for a larger version):
More documentation for the client API can be found in the
Administrator's Guide (look in the UI
Templates section) and the
GalleryServerPro.Web.Entity
API documentation.
Overview
Rendering JPG images in a browser is straightforward because all browsers use the same syntax based on the
<img>
HTML tag. But video, audio, and other file types have wildly different levels of support in various browsers and within
different versions of the same browser. For example, some browsers can natively play H.264 MP4 video while others require
a plug-in such as Flash or Silverlight.
Gallery Server Pro solves this problem with a set of media templates that can deliver the correct HTML and JavaScript for
individual browsers or even versions of browsers. A default set of templates are included that will work well for most users,
but some organizations may require different behavior. Or, when a new browser version is released, its support for a particular
media type may change, requiring a change in the HTML syntax that is used.
The media templates page lets one manage the rendering behavior of different media types.
The above screen shot shows the media template used for the 'default' browser when rendering a media file having MIME type
video/mp4. Notice it specifies the HTML5 <video> tag. When MP4 videos are viewed in most browsers, the HTML is based
on this template, as seen here in Safari:
The Browser ID dropdown allows one to view media templates for other browsers. For example, the MP4 templates includes two
additional templates to support IE 1-8 and Opera, which do not support the HTML5 video tag:
Notice the HTML template for IE 1-8 and Opera are identical and consists of a hyperlink tag. This defines the HTML for using
FlowPlayer, which uses the Flash plug-in to render the video. FlowPlayer requires some JavaScript to initialize it, which
you can see when you click the JavaScript tab:
When an MP4 video is viewed in IE 1-8 or Opera, it gets HTML and JavaScript based on the corresponding template, as seen
in this screen shot from Opera:
Through the use of custom media templates for individual browsers, Gallery Server Pro can provide targeted behavior without
requiring complicated HTML containing fallback code and other tricks. The Administrator's Guide has more information
about editing and creating media templates.
Overview
FFmpeg is an open source utility that can transcode video and audio and extract
thumbnail images from videos. When ffmpeg.exe is present in the bin directory, Gallery Server Pro automatically uses it
to create web-optimized versions of video and audio files. This can result in dramatically smaller files and an improved
user experience.
FFmpeg can be downloaded from the official FFmpeg website. Since the site posts only the
source code, you have to compile
it yourself. For convenience, we've done that part and packaged it into a
free download called the Gallery Server Pro Binary
Pack. Although we won't talk about them here, the package includes two additional open source tools—ImageMagick
and GhostScript—which are used to generate
thumbnails from certain image and document types.
Using FFmpeg is easy. For example, one can create a Flash Video file from a Windows Media File video by executing the
following at a command prompt:
ffmpeg.exe -i "C:\myvideo.wmv" "C:\myvideo.flv"
FFmpeg supports a large number of options that control how the output video or audio file is created. In Gallery Server
Pro, we want to create H.264 MP4 web-optimized videos that will play in as many devices as possible, including mobile
devices. We came up with the following:
ffmpeg.exe -y -i "{SourceFilePath}" -vf "scale=min(iw*min(640/iw\,480/ih)\,iw):min(ih*min(640/iw\,480/ih)\,ih)"
-b:v 384k -vcodec libx264 -flags +loop+mv4 -cmp 256 -partitions +parti4x4+parti8x8+partp4x4+partp8x8 -subq 6 -trellis 0
-refs 5 -bf 0 -coder 0 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -qmin 10 -qmax 51 -qdiff 4
-ac 1 -ar 16000 -r 13 -ab 32000 -movflags +faststart "{DestinationFilePath}"
That's a real handful to understand and we won't disect the entire thing, but let's look at a few key things.
-vf
specifies a filter that resizes the video to approximately 640px by 480px while preserving the aspect ratio. -vcodec
specifies the libx264 codec. -movflags uses the faststart option that results in a video that can be played after a
small amount of buffering. Without this flag, the entire video must be downloaded before it begins playback.
Learn more about these options by reading the FFmpeg documentation. The
options used in Gallery Server Pro can be viewed and changed on the Video & Audio page in the site admin area.
Calling FFmpeg from ASP.NET
As we just saw, we can use FFmpeg to transcode a video or audio file at a command prompt. To invoke it from ASP.NET, we
use System.Diagnostics.Process:
private void Execute()
{
bool processCompletedSuccessfully = false;
InitializeOutput();
var info = new ProcessStartInfo(AppSetting.Instance.FFmpegPath, MediaSettings.FFmpegArgs);
info.UseShellExecute = false;
info.CreateNoWindow = true;
info.RedirectStandardError = true;
info.RedirectStandardOutput = true;
using (Process p = new Process())
{
try
{
p.StartInfo = info;
p.ErrorDataReceived += ErrorDataReceived;
p.Start();
p.BeginErrorReadLine();
processCompletedSuccessfully = p.WaitForExit(MediaSettings.TimeoutMs);
if (!processCompletedSuccessfully)
p.Kill();
p.WaitForExit();
if (!processCompletedSuccessfully || MediaSettings.CancellationToken.IsCancellationRequested)
{
File.Delete(MediaSettings.FilePathDestination);
}
}
catch (Exception ex)
{
if (!ex.Data.Contains("args"))
{
ex.Data.Add("args", MediaSettings.FFmpegArgs);
}
Events.EventController.RecordError(ex, AppSetting.Instance, MediaSettings.GalleryId, Factory.LoadGallerySettings());
}
}
}
private void InitializeOutput()
{
var item = MediaConversionQueue.Instance.GetCurrentMediaQueueItem();
if ((item != null) && (item.MediaQueueId == MediaSettings.MediaQueueId))
{
_output.Append(item.StatusDetail);
}
IMediaEncoderSettings mes = MediaSettings.EncoderSetting;
if (mes != null)
{
_output.AppendLine(String.Format("{0} => {1}; {2}", mes.SourceFileExtension, mes.DestinationFileExtension, mes.EncoderArguments));
}
_output.AppendLine("Argument String:");
_output.AppendLine(MediaSettings.FFmpegArgs);
}
private void ErrorDataReceived(object sender, DataReceivedEventArgs e)
{
_output.AppendLine(e.Data);
var item = MediaConversionQueue.Instance.GetCurrentMediaQueueItem();
if ((item != null) && (item.MediaQueueId == MediaSettings.MediaQueueId))
{
item.StatusDetail = Output;
}
CancelIfRequested(sender as Process);
}
private void CancelIfRequested(Process process)
{
CancellationToken ct = MediaSettings.CancellationToken;
if (ct.IsCancellationRequested)
{
if (process != null)
{
try
{
process.Kill();
process.WaitForExit();
}
catch (Win32Exception) { }
catch (SystemException) { }
}
}
}
The full path to the ffmpeg.exe executable and the command line arguments are passed into an instance of ProcessStartInfo
,
which is then assigned to the StartInfo
property of a Process
instance. As FFmpeg
processes the file, it periodically returns data, which is collected in a StringBuilder
in the ErrorDataReceived
event and ultimately logged to the event log.
A timeout value is passed to the WaitForExit
method to automatically shut down the process if it doesn't
complete in a reasonable time.
Killing the process
There are a couple situations where we want to kill the process before it completes on its own:
- The user deletes the media object being processed.
- The user deletes the media queue item on the Video & Audio page in the site admin area.
Each time FFmpeg calls the ErrorDataReceived
event to update it with a progress report, we check a
cancellation token to see if the flag has been set to request a cancellation. If it has, we call the Kill
method as shown in the code snippet above.
But how is the cancellation token assigned and updated? The processing of media items with FFmpeg is managed by a singleton
class named MediaConversionQueue
. That
class has a property called CancelTokenSource:
protected CancellationTokenSource CancelTokenSource { get; set; }
Each time we ask FFmpeg to do a conversion, we assign this property and pass its Token
property
to the FFmpeg
class:
CancelTokenSource = new CancellationTokenSource();
var mediaSettings = new MediaConversionSettings
{
CancellationToken = CancelTokenSource.Token
};
mediaSettings.FFmpegOutput = FFmpeg.CreateMedia(mediaSettings);
This token is what we are checking in the ErrorDataReceived
event. When a media object is deleted or the
queue item is removed, the Cancel
method is invoked on the token source (in MediaConversionQueue.RemoveMediaQueueItem
):
CancelTokenSource.Cancel();
Calling the Cancel
method causes subsequent calls to the IsCancellationRequested
property to return
true
,
triggering the code in the ErrorDataReceived
event to kill the process.
Each media object (photo, video, etc.) is stored in an album. Albums can be nested within other albums, with
no restriction on the number of levels. This is similar to how files and directories are stored on a hard drive.
It turns out that albums and media objects have a lot in common. They both have properties such as Id
, Title
,
DateAdded
, and FullPhysicalPath
; and they both have methods such as Save
and
Delete
. This is the ideal situation
in which to use the composite design pattern, where common functionality is defined in a base object. I
start
by defining two interfaces — IGalleryObject
and IAlbum
:
The IAlbum
interface inherits from IGalleryObject
and then adds a few methods and properties
that
are specific to albums. Then I create the abstract
base class GalleryObject
. It implements the
IGalleryObject
interface and provides default behavior that is common to albums and media objects. For example,
here is the DateAdded
property:
public DateTime DateAdded
{
get
{
VerifyObjectIsInflated(this._dateAdded);
return this._dateAdded;
}
set
{
this._hasChanges = (this._dateAdded == value ? this._hasChanges : true);
this._dateAdded = value;
}
}
Now that the common functionality is defined in the abstract
base class, I can create concrete classes to represent
albums, images, video, audio, and other types of media objects:
With this approach, there is very little duplicate code, the structure is maintainable, and it is easy to work with. For
example, when Gallery Server Pro wants to display the title and thumbnail image for all the objects in an album, there might
be any combination of child albums, images, video, audio, and other documents. But I don't need to worry about all the different
classes or about casting problems. All I need is the following code:
IAlbum album = Factory.LoadAlbumInstance(42, true);
foreach (IGalleryObject galleryObject in album.GetChildGalleryObjects())
{
string title = galleryObject.Title;
string thumbnailPath = galleryObject.Thumbnail.FileNamePhysicalPath;
}
Beautiful, isn't it? But what happens when the functionality is slightly different between two types of objects? For example,
Gallery Server Pro needs to verify that each media object has a thumbnail image, but albums do not have a
thumbnail image, at least not in the strict sense of the word. (The thumbnail image you see for albums is actually the
thumbnail of one of its media objects.) We want to define this validation in the base class so that it is inherited by
all concrete classes, but one of the concrete classes—Album
—requires different validation behavior.
We solve this by first creating the validation
function in the GalleryObject
class:
protected virtual void CheckForThumbnailImage()
{
if (!System.IO.File.Exists(this.Thumbnail.FileNamePhysicalPath))
{
this.RegenerateThumbnailOnSave = true;
}
}
Since it is defined as protected virtual
, we can override
it in a derived class. In fact, that is exactly what the
Album
class does:
protected override void CheckForThumbnailImage()
{
}
The end result is we have an implementation in the base class that provides functionality for most cases and
code
that is unique to albums is contained in the Album
class. There isn't any duplicate code and the logic is nicely
encapsulated. It is a thing of beauty to behold.
We just saw how to override a function when we need to alter its behavior. We could have done something similar
when it comes to saving the albums and media objects to the database. The Save
method in the GalleryObject
class could have been defined as virtual, and we could have overridden the method in each of the derived classes. But since
the classes Image
, Video
, Audio
, GenericMediaObject
, and ExternalMediaObject
all represent objects that get stored in the same table (MediaObject
), that would have meant writing the
same code in all five classes, with only the Album
class being different.
One approach that eliminates the problem of duplicate code is to provide a default implementation in the Save
method in
the GalleryObject
class. In that method, I save to the media object table, and then depend on the Album
class to override the behavior, much like we did with the CheckForThumbnailImage
function. However, this is
putting a
substantial amount of behavior in a base class that doesn't really belong there. We should limit the base class to contain
state and behavior that applies to ALL derived objects.
You might argue that I violated this rule when I provided a default implementation of the CheckForThumbnailImage
method
that I overrode in the Album
class. You are absolutely right. But I justify it by suggesting that implementing
the validation in every derived class creates undesirable duplicate code, and refactoring it to use the strategy pattern
is overkill. These are not hard and fast rules. Architecting an application is as much art as it is science, and you must
weigh the pros and cons of each approach.
Getting back to our challenge of persisting data to the data store, the approach we came up with was to use the strategy
pattern to encapsulate behavior. First, I defined an interface ISaveBehavior
:
public interface ISaveBehavior { void Save(); }
Then I wrote two classes that implemented the interface: AlbumSaveBehavior
and MediaObjectSaveBehavior
.
The Save
method takes care of persisting the object to the hard drive and data store. For example, here is
the Save
method in AlbumSaveBehavior
:
public void Save()
{
if (this._albumObject.IsVirtualAlbum)
return;
PersistToFileSystemStore(this._albumObject);
using (var repo = new AlbumRepository())
{
repo.Save(this._albumObject);
}
if (this._albumObject.GalleryIdHasChanged)
{
AssignNewGalleryId(this._albumObject.GetChildGalleryObjects(GalleryObjectType.Album), this._albumObject.GalleryId);
}
}
We'll skip showing the implementation from the MediaObjectSaveBehavior
class, but it contains logic that
is
specific to persisting media objects.
OK, we have two classes for saving data to the data store—one for albums and one for media objects. How do we invoke
the
appropriate Save
method from the GalleryObject
base class?
Recall that the GalleryObject
class is abstract
, so it can never be directly instantiated. Instead,
we instantiate an instance of the Album
, Image
, Video
, Audio
, GenericMediaObject
,
or ExternalMediaObject
class. The constructor for each of these classes assigns the appropriate save behavior.
For example, in the constructor of the Album
class, we have:
this.SaveBehavior = Factory.GetAlbumSaveBehavior(this);
The GetAlbumSaveBehavior
method just returns an instance of the AlbumSaveBehavior
class:
public static ISaveBehavior GetAlbumSaveBehavior(IAlbum albumObject)
{
return new AlbumSaveBehavior(albumObject);
}
The SaveBehavior
property of the GalleryObject
class is of type ISaveBehavior
. Since
both classes implement this interface, we can assign instances of either class to the property.
The Save
method in the GalleryObject
class simply calls the Save
method on the
SaveBehavior
property. It has no idea whether the property is an instance of
AlbumSaveBehavior
or
MediaObjectSaveBehavior
, and it doesn't care. All that matters is that each class knows how to save its designated
object.
This is an example of using the strategy pattern. Specifically, the strategy pattern is defined as a family of algorithms
that are encapsulated and interchangeable. In our case, we have two save behaviors that are self-contained and can both
be assigned to the same property (interchangeable). It is a powerful pattern and has many uses.
This has been a brief introduction to the architecture and programming techniques used in Gallery Server Pro. Feel free
to
download the source code and use the bits to help in your own project. Cheers!
Article History
- 2013 Oct 18
- Updated for version 3, including several new sections
- Updated source code to 3.0.3
- 2011 July 1
- 2011 Apr 27
- Updated source code to 2.4.7
- Updated article to reflect changes since last update
- 2008 Nov 3
- Updated source code to 2.1.3222
- 2008 Sep 9
- Updated to include latest source files, references to new features, and minor content updates
- 2008 May 6
- Updated to include latest source files and minor content updates
- 2007 Oct 28