Creating and Installing Plugins as Nuget Packages

Introduction

Most Important Tasks of Software Architects

The most important software development tasks of the software architects are the following:

1. Create and maintain the Plugin Infrastructure, allowing the individual developers to create, test, debug and extend plugins almost independently of each other. The Plugin Infrastructure should take care of
   • hosting the plugins
   • arranging, displaying, saving and restoring the layout of the visual plugins
   • allowing to easily mock the plugins which are not ready yet
2. Finding and factoring out the code and concepts that can be re-used in multiple places.

Of course, the architects also need to be gathering requirements, estimating how long implementing a feature would take, interacting with the clients, choosing the software and testing frameworks and so on, but above, we are talking only about the 'software development' tasks.

Minimal Plugin Interfaces and Plugin Framework

In Generic Minimal Inversion-of-Control/Dependency Injection Interfaces implemented by the Refactored NP.IoCy and AutofacAdapter Containers, I present minimal interfaces for an IoC (Plugin) container and their implementation as NP.IoCy plugin framework as well as AutofacAdapter - an implementation built on top of Autofac.

The multiplatform implementation for Visual Plugins - Gidon - Avalonia based MVVM Plugin IoC Container is still very much in progress and when completed, will allow

1. hosting Python Shells and Visual Pages
2. hosting C# scripting Shells and Visual Pages
3. hosting Web Pages

All of these will be working on multiple desktop platforms (Windows, Linux and Mac).

Installing the Plugins

There is a question of how to install the plugins into an application.

It turns out that each plugin can be turned into a special nuget package and then installed as a nuget package with some simple special processing at installation time.

The plugin package creation and installation should follow the following principles:

1. The plugin NuGet should include all the DLLs - the main plugin DLL and those it depends on (including the nuget dependencies).
2. The main project should not depend on the plugin DLLs - all the plugin DLLs should be dynamically loaded.

Nuget Documentation (or Lack of It)

As many of you might have learned, it is very difficult to find information about packaging and unpackaging files as a nuget package whether using nuget command or csproj files and MSBuild command.

Out of many web pages I tried, only two were really useful:

1. Tips & Tricks to improve your .NET Build Setup with MSBuild with Github samples located on rider-msbuild-webinar-2020. I did not watch it all - just section 6.
2. How to find a NuGet package path from MSBuild explaining how to turn a path inside a nuget package into a csproj variable and then copy files out of package using that path into a disk folder you choose.

The new versions of MSBuild and csproj match or exceed all functionality nuget utility, so, I am moving all my packages (including plugin packages) to be built without nuget and without nuspec files simply by building the corresponding C# project in Visual Studio.

Code Location

The sample code is located within NP.Samples repository under PluginPackageSamples folder.

Packaging/Unpackaging Samples

Our Packaging/Unpackaging samples are based on the Multiple Plugins Test from the Generic Minimal Inversion-of-Control/Dependency Injection Interfaces implemented by the Refactored NP.IoCy and AutofacAdapter Containers article.

Brief Explanations of what the Plugins Do

Plugins are pieces of software that can be loaded dynamically into the framework. Plugins should not depend on each other and should not depend on the Plugin Framework, instead, they can be dependent on a set of common interfaces and communicate via those interfaces. Such Plugin infrastructure will increase the separation of concerns and allow Plugins to be developed, debugged and extended almost independently of each other and of the plugin framework.

Here, we give only a brief explanation of what our test plugins do. For full explanations of the plugins' implementations, please look at the Multiple Plugins Test link.

There are two very simple plugins involved:

1. DoubleManipulationPlugin - providing two methods for manipulating doubles: Plus(...) for summing up two numbers and Times(...) for multiplying two numbers.
2. StringManipulationPlugins also providing two methods for string manipulations: Concat(...) - for concatenating two strings and Repeat(...) for repeating a string several times.

These two plugins do not depend on each other and the main project does not have a dependency on them. Instead, both plugins and the main project have a dependency on the project PluginInterfaces containing two interfaces, one for each of the plugins:

public interface IDoubleManipulationsPlugin
{
    double Plus(double number1, double number2);

    double Times(double number1, double number2);
}

public interface IStringManipulationsPlugin
{
    string Concat(string str1, string str2);

    string Repeat(string str, int numberTimesToRepeat);
}

These interfaces are defined within common project NP.PackagePluginsTest.PluginInterfaces located within PluginInterfaces folder.

Packaging Sample

Solution PackagePluginsTest.sln (that creates the plugins as nuget packages) is located inder PluginPackageSamples\PackagePlugins folder. It contains three projects:

1. NP.PackagesPluginsTest.DoubleManipulationsPlugin for creating the NP.PackagesPluginsTest.DoubleManipulationsPlugin.nupkg package
2. NP.PackagesPluginsTest.StringManipulationsPlugin for creating the NP.PackagesPluginsTest.StringManipulationsPlugin.nupkg package
3. NP.PackagePluginsTest.PluginInterfaces whose reference is shared between the above projects.

Let us take a look at NP.PackagesPluginsTest.DoubleManipulationsPlugin project (the other one is very similar except that its methods are different and refer to manipulating with strings instead of doubles).

DoubleManipulationsPlugin implements IDoubleManipulationsPlugin interface by defining two methods - double Plus(double number1, double number2) and double Times(double number1, double number2).

The class is marked with RegisterTypeAttribute so that the NP.IoCy framework would know how to read it and register it within its container:

[RegisterType]
public class DoubleManipulationsPlugin : IDoubleManipulationsPlugin
{
    // sums two numbers
    public double Plus(double number1, double number2)
    {
        return number1 + number2;
    }

    // multiplies two numbers
    public double Times(double number1, double number2)
    {
        return number1 * number2;
    }
}  

Now take a look at the csproj file of the project - NP.PackagePluginsTest.DoubleManipulationsPlugin.csproj.

Within the <PropertyTag> tag at the top, we add some nuget package properties (including the Version - 1.0.4, Copyright - Nick Polyak 2023, PackageLicenseExpression - MIT).

The most important properties defined there are:

1. <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies> - This property will copy all the DLL files (including those coming from dependent nuget packages) into the output folder to become part of the package.
2. <GeneratePackageOnBuild>true</GeneratePackageOnBuild> - This property creates the nuget package everytime the project is built. The created .nupkg file will be located one folder above the target folder (for example, if the target folder is within bin/Debug/net6.0 under the project folder, then the .nupkg file will be within bin/Debug folder).

After the PropertyGroup, there are two references - one PackageReference - to the NP.DependencyInjection project got the IoC attributes and another - a project reference to the already mentioned NP.PackagePluginsTest.PluginInterfaces project to get the interfaces which we implement.

At the end of the file - there are two Target tags:

<Target Name="ClearTarget" BeforeTargets="Build">
	<RemoveDir Directories="$(TargetDir)\**" />
</Target>

<Target Name="IncludeAllFilesInTargetDir" AfterTargets="Build">
	<ItemGroup>
		<Content Include="$(TargetDir)\**">
			<Pack>true</Pack>
			<PackagePath>Content</PackagePath>
		</Content>
	</ItemGroup>
</Target>  

The first target is fired before the build (BeforeTargets="Build") and removes all files or subfolders from the $(TargetDir).

The second target is fired after the build. It creates the nuget package by including all the files and sub-folders from the target directory within Nuget package's Content folder.

Here is the content of the resulting plugin viewed using NuGetPackageExplorer:

All the files are contained under Content folder of the nuget package and only one - NP.PackagePluginsTest.DoubleManipulationPlugin.dll is container within the usual place - lib/net6.0. I do not know how to get rid of that last file there but if I knew the consuming side changes would have been a little simpler (nothing radical though).

After you create the two nuget package files - you have to upload them over to either nuget.org or some other nuget server (e.g., a local server). I already uploaded the two plugin files, NP.PackagePluginsTest.DoubleManipulationsPlugin.1.0.4.nupkg and NP.PackagePluginsTest.StringManipulationsPlugin.1.0.4.nupkg onto nuget.org server. So you do not have to do it - you can simply use my files that exist on the nuget.org already.

Package Consumption Sample

The sample that shows how to create a plugin out of the uploaded nuget files is located within PluginPackageSamples\PluginConsumer\PluginConsumer.sln solution. The solution has two projects:

1. PluginsConsumer - The main project that downloads the nuget package, creates plugins out of them and uses them to provide the implementations for the interfaces.
2. NP.PackagePluginsTest.PluginInterfaces - the project containing common interfaces

The content of Program.cs file is almost the same as the content of the main file described in Multiple Plugins Test section of the previous article. Because of that, I'll describe only the beginning of the main program where the plugins are being dynamically loaded, IoC container is created and doublemanipulatesPlugin is resolved from the IoC container:

// create container builder
IContainerBuilder<string?> builder = new ContainerBuilder<string?>();

// load plugins dynamically from sub-folders of Plugins folder
// located under the same folder as the executable
builder.RegisterPluginsFromSubFolders("Plugins");

// build the container
IDependencyInjectionContainer<string?> container = builder.Build();

// get the plugin for manipulating double numbers
IDoubleManipulationsPlugin doubleManipulationsPlugin =
    container.Resolve<IDoubleManipulationsPlugin>();

// get the result of 4 * 5
double timesResult =
    doubleManipulationsPlugin.Times(4.0, 5.0);

// check that 4 * 5 == 20
timesResult.Should().Be(20.0);  

The key line here is builder.RegisterPluginsFromSubFolders("Plugins"); which attempt to dynamically load the plugins from all subfolders of the $(TargetDir)/Plugins.

The most interesting code is within the csproj file PluginsConsumer.csproj. This file has an ItemGroup containing package references to all packages including the plugin packages:

<ItemGroup>
    ...
    <PackageReference Include="NP.PackagePluginsTest.DoubleManipulationsPlugin" 
     Version="1.0.4" GeneratePathProperty="true">
        <ExcludeAssets>All</ExcludeAssets>
    </PackageReference>
    <PackageReference Include="NP.PackagePluginsTest.StringManipulationsPlugin" 
     Version="1.0.4" GeneratePathProperty="true">
        <ExcludeAssets>All</ExcludeAssets>
    </PackageReference>
</ItemGroup>  

Note that both packages exclude all assets. This is done so that the main program would not statically depend on the NP.PackagePluginsTests.DoubleManipulationsPlugin.dll and NP.PackagePluginsTests.StringManipulationsPlugin.dll files and so that they would be dynamically loaded (same as the rest of the DLL assemblies).

Also, note that both PackageReferences to the plugins have GeneratePathProperty="true" attribute. This attribute automatically generates a variable set to the path of the plugin root. The variable name is always prefixed with "Pkg" and every period within the package name is replaced by an underscore - "_". For example, the variable name for NP.PackagePluginsTest.DoubleManipulationPlugin root folder will be PkgNP_PackagePluginsTest_DoubleManipulationPlugin.

These variable names we use in the next ItemGroup, where we set all files and subfolders that need to be copied from the packages:

<ItemGroup> <!-- setting up the variable for convenience -->
    <DoubleManipPluginPackageFiles 
     Include="$(PkgNP_PackagePluginsTest_DoubleManipulationsPlugin)\Content\**\*.*" />
    <StringManipPluginPackageFiles 
     Include="$(PkgNP_PackagePluginsTest_StringManipulationsPlugin)\Content\**\*.*" />
</ItemGroup>  

Now we come to the Target that copies the files from <nuget_package_root>\Content to $(TargetDir)/Plugins/<PluginName> folder after the build:

<Target Name="CopyPluginsFromNugetPackages" AfterTargets="Build">
    <PropertyGroup>
        <DoublePluginFolder>$(TargetDir)\Plugins\DoubleManipulationPlugin
        </DoublePluginFolder>
        <StringPluginFolder>$(TargetDir)\Plugins\StringManipulationPlugin
        </StringPluginFolder>
    </PropertyGroup>
    <RemoveDir Directories="$(DoublePluginFolder)" />
    <Copy SourceFiles="@(DoubleManipPluginPackageFiles)" 
     DestinationFolder="$(DoublePluginFolder)%(RecursiveDir)" />
    <RemoveDir Directories="$(StringPluginFolder)" />
    <Copy SourceFiles="@(StringManipPluginPackageFiles)" 
     DestinationFolder="$(StringPluginFolder)%(RecursiveDir)" />
</Target>  

At the top of the Target tag, we define the variables DoublePluginFolder and StringPluginFolder used in multiple places later within the same target:

<PropertyGroup>
    <DoublePluginFolder>$(TargetDir)\Plugins\DoubleManipulationPlugin
    </DoublePluginFolder>
    <StringPluginFolder>$(TargetDir)\Plugins\StringManipulationPlugin
    </StringPluginFolder>
</PropertyGroup> 

Then, for each of the plugins, we:

1. First - remove the plugin folder, e.g., <RemoveDir Directories="$(DoublePluginFolder)" />
2. Then - copy the files from the nuget package to the Plugin folder, e.g.: <Copy SourceFiles="@(DoubleManipPluginPackageFiles)" DestinationFolder="$(DoublePluginFolder)%(RecursiveDir)" />

%(RecursiveDir) at the end of the path allows us to preserve the same folder structure as within the plugin's nuget package file. Sometimes, it is important, e.g., when the package has a runtime subfolder with multiple folders corresponding to each native platform within it.

Now you should be able to build PluginConsumer project and to run it. After you build it, verify that you have Plugins folder within bin\Debug\net6.0 folder and that folder has two subfolders, DoublePluginFolder and StringPluginFolder, each populated with the corresponding plugin file. Running the project without errors will prove that the container really has those dynamically loaded plugins and they resolve correctly to the corresponding interfaces and that all the plugin functionality is working.

Note that all the plugin-related addition to csproj files are preserved when you, e.g., upgrade the version of the corresponding plugin via Visual Studio. The only time when you need to edit the csproj files is when a plugin is added or removed.

Those who read the code thoroughly, might notice that at the end of Program.cs file, I am testing some cryptic "MethodNames":

var methodNames = container.Resolve<IEnumerable<string>>("MethodNames");

methodNames.Count().Should().Be(4);

methodNames.Should().Contain(nameof(IDoubleManipulationsPlugin.Plus));
methodNames.Should().Contain(nameof(IDoubleManipulationsPlugin.Times));
methodNames.Should().Contain(nameof(IStringManipulationsPlugin.Concat));
methodNames.Should().Contain(nameof(IStringManipulationsPlugin.Repeat)); 

For those interested in what's going on there, please read about MultiCells in the previous article - Multi-Cells and Plugins with Multi-Cells.

Conclusion

This article dealt with creating and installing plugins as nuget packages. This allows us to mostly rely on MSBuild functionality embedded into Visual Studio for creating and installing the plugins. Essentially, we do not need to create any (or almost any) special installation mechanisms for creating and installing the plugins.

I plan to use this method of creating and installing the plugins in my future articles adding plugins to Google RPC server.

History

• 17^th January, 2023: Initial version