Click here to Skip to main content
15,867,453 members
Articles / Programming Languages / ASM

Integrating a compiler/assembler in VS ; Using NASM with Visual Studio 2010

Rate me:
Please Sign up or sign in to vote.
5.00/5 (11 votes)
9 Jul 2012CPOL10 min read 93.6K   2.9K   29   21
A simple technique to integrate a custom build tool in VS 2010.

Assembly in Action

Introduction

For a long time it has been seen in various assembly forums, users asking for integration of their tools in Visual Studio as well as in other IDEs. Although there is already a couple of them coming bundled with VS (like MASM and LICX), for the rest appears just a wish list. After the VS 2010 release, the old rules files have become obsolete and for enhanced customization, XML based task factory acquired the center stage. This configuration in comparision to the rules is a highly customized target-task entity that invokes and executes the handling code after setting properties on it. Such an approach puts a choice to developer to integrate their own tools and perform atomic build operations on a specific code. In the demonstrated article whole build operation for an assembly file is driven from a set of triple configurations i.e. an XML (containing display and options configuration [Property Page UI]),a target file (containing strategy for the build system) and a property sheet (for initialization and declarations).

This article only covers minimal/basic concepts to write a target file for VS. For an advanced configuration you in fact have to refer MSDN documentation. To keep the simplicity for beginners I will adhere to describing things in chunks using analogical and functional approach, hence I request veterans to endure this and to also suggest better methodologies.

Content Index

The article has the following objectives:

  1. Requirement gathering.
  2. Available technical options.
  3. Understanding the selected option.
  4. Creation and Implementation.

Requirement Gathering

The requirements are as follows:

  • Enable a NASM assembly file to be recognized in Visual Studio 2010 or above
  • Incorporate assembly code for utilization with other compatible projects like C, C++ project.
  • A GUI based Property Page to enter parameter for build Operation.

Available Technical Options

At this stage, to meet the requirement, adding an option either in "New Project" with a new sort of project or in "Build customizations", is visible choice. Going with New Project is a more cumbersome task and to do it MSDN already has got walkthroughs. Also it requires more effort to make it compatible with other languages compilation. Build customizations are simpler in comparison and thus the selection.

Understanding the Selected Option

Revisiting the requirement gathering section, it is now observable that the first two requirements are already met with the build customizations option [First one by default in VS]. For the third one it is good to start by first creating three files namely nasm.props, nasm.targets, and nasm.xml. These files will determine the layout, sequences and the fate of the build operation. Anatomy, physiology and morphology of files are individually illustrated.

nasm.xml

This XML acts as delegate to render UI controls in the property page. It has got collection of tags with a set of predefined attributes whose set values are passed as parameter to the UI designer which in accordance layouts the interface. A minimal functional configuration is shown below followed by a table for the explanation of included components.

XML
<?xml version="1.0" encoding="utf-8"?>
<ProjectSchemaDefinitions xmlns="http://schemas.microsoft.com/build/2009/properties" 
   xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
   xmlns:sys="clr-namespace:System;assembly=mscorlib">
   <Rule Name="NASM" PageTemplate="tool" 
    DisplayName="Netwide Assembler" Order="200">
 <Rule.DataSource>
 <DataSource Persistence="ProjectFile"   ItemType="NASM" />
 </Rule.DataSource>
 <Rule.Categories>
 <Category   Name="General">
    <Category.DisplayName>
  <sys:String>General</sys:String>
    </Category.DisplayName>
 </Category>
 </Rule.Categories>
 <StringProperty Name="ExecutionDescription" 
  DisplayName="Execution Description" IncludeInCommandLine="False" Visible="False" />
 <EnumProperty Name="Outputswitch" Category="Assembler Options" 
     HelpUrl="http://www.nasm.us/doc/"   DisplayName="Output Switch" 
     Description="Select the type of output format required. 
        Linking Should be disabled for ELF and Binary ,else error will popup">
 <EnumValue Name="0" DisplayName="Object File" Switch="-fwin" />
 <EnumValue Name="1" DisplayName="LINUX ELF FORMAT" Switch="-f elf" />
 </EnumProperty>
 <StringListProperty Name="AssembledCodeListingFile" Category="Assembler Options" 
    DisplayName="Assembled Code Listing File" Description="Generates an assembled code listing file. (-l [file])" 
    HelpUrl="http://www.nasm.us/doc/" Switch="-l "[value]"" />
 <BoolProperty Name="GenerateDebugInformation" Category="Assembler Options" 
   DisplayName="Generate Debug Information" 
   Description="Generates Debug Information. (-g)" 
   HelpUrl="http://www.nasm.us/doc/" Switch="-g" />
<! -- Optionally shown for illustration>
 <IntProperty Name="SectionAlignment" DisplayName="SectionAlignment" 
    Description="The /ALIGN option specifies the alignment of each section within the 
       linear address space of the program. The number argument is in bytes and must be a power of two." 
    Category="Advanced" Switch="ALIGN" Separator=":" 
    F1Keyword="VC.Project.VCLinkerTool.Alignment">   </IntProperty>
 <DynamicEnumProperty   Name="NASMBeforeTargets" Category="General" 
    EnumProvider="Targets" IncludeInCommandLine="False">
 <DynamicEnumProperty.ProviderSettings>
     <NameValuePair Name="Exclude" Value="^NASMBeforeTargets|^Compute" />
 </DynamicEnumProperty.ProviderSettings>
 <DynamicEnumProperty.DataSource>
    <DataSource Persistence="ProjectFile" ItemType="" HasConfigurationCondition="true" />
 </DynamicEnumProperty.DataSource>
 </DynamicEnumProperty>
   </Rule>
   <ItemType Name="NASM" DisplayName="Netwide Assembler" />
   <FileExtension Name="*.asm" ContentType="NASM" />
   <ContentType Name="NASM" DisplayName="Netwide Assembler" ItemType="NASM" />
</ProjectSchemaDefinitions>

Image 2

Tag/Attribute

Application

Remarks

ProjectSchemaDefinitions

Represents a data-driven project schema.

Parent tag where the rule and other definitions are embedded.
Rule It initializes the definition and also contains a few essential attribute which molds the behavior of other properties like SwitchPrefix.
Rule.DataSource This indicates to the item/location from where the property has to be read and written to. Its behavior is further decidable by the persistence and Itemtype attribute.
Rule.Categories The Category tags are defined under this tag.
Category Adding into this node creates a new value in the left pane of property page.An approximate HTML illustration has been shown to immitate the view.
General (Category1)
C/C++ (Category2)
Linker (Category3)
OptimizeBoolProperty
Output EnumProperty
Filename StringListProperty
StringProperty This tag creates editable text box that can be multiline.
EnumProperty This creates dropdown box populated with the values in Enum tag.
StringListProperty A simple Textbox control which has value <Edit> to pop up another textbox.
BoolProperty Dropdown list with a Yes/No Option
DynamicEnumProperty Dynamically populated values, user intervention not necessary
NameValuePair More like dictionary implementation where properties are set using name and value pair. Name cannot be empty.
DynamicEnumProperty. ProviderSettings Sets a provider-specific set of options to pass to the provider.
IntProperty Used to set integer values. It is also a textbox control.
ItemType Indicates whether the properties will be stored as ItemDefinition metadata or Item metadata (depends on how the property page is summoned i.e. from menu options or from solution explorer) of this item type. If this field is not set, then the property is written as a common property in a PropertyGroup.
FileExtension What kind of files are included to be built with this configuration. Use ; to separate extensions
ContentType Maps the files to certain itemtype.
SwitchPrefix Although not visible in above sample code, it prefixes its value in every switch which makes its way to command line. Good shortcut for early definition.
Its syntax is
<Rule Name="CL" PageTemplate="tool" SwitchPrefix="/" Order="10"></Rule>

<rule switchprefix="/">
HasConfiguration Simply a true value indicates the property’s value to be imposed on the whole project, else it could be set for individual files in the project.
PageTemplate Puts a choice for the UI template from UI Template collection. Like a tool template renders property page right pane in grid format. For a clear picture open the property page of a native C++ project. Under Configuration Properties category take a look on
+ General (PageTemplate:generic)
+ Debugging (PageTemplate: debugger)
+ C/C++ (PageTemplate:tool)
Category Signifies the category of this property where it has to be placed and shown.
Persistence ProjectFile causes the property value to be written to and read from the project manifest file or the property sheet, depending on which node in Solution Explorer or the Properties window is used to spawn the property pages UI. UserFile causes the property value to be written to and read from the .user file. This field is mandatory and is culture-invariant. Current accepted values are ProjectFile and UserFile.
Visible Implies whether the control should appear on UI or not.
DispalyName Anything written inside this field appears directly as screen name in the property page. Values Optimize,Output,Filename in Categories row are the example of DisplayName.
Description This appears at the bottom of the right pane whenever we click a property.
Switch The main culprit for which this whole mesh has to be created.
It will pass values to the command line according to the selected XXXProperty.
IncludeInCommandLine Yes indeed the name implies the right thing, it decides the inclusion in command line parameter.
HelpUrl It’s the user’s choice to provide some help URL for the property.
Order Relative location where the Node(i.e. Netwide Assembler) will appear corresponding to other categories in property page.
__________________________________________________
Higher the order, lower the location.

nasm.targets:-

Another XML file which packages the information for target, task and other definitions.

A simple target file is composed of:

+ Item hierarchy that includes (from parent to child):-

+ Targets hierarchy

+ Property hierarchy

The above components are utilized in build process. The basic functional unit is the task element which is encapsulated inside a target element. Each target element align in an order in the target file so as they can be executed in sequence and so are the task elements which in turn inside the target element arranged to execute sequentially in order of their corresponding position. MSBuild properties and items are both used to pass information to tasks, evaluate conditions, and store values that can be referenced throughout the project file. Tasks provide the code that runs during the build process. Also every element has a set of attributes that defines the element’s behavior.

Below is the tag by tag explanation of the whole target file (shown in comment):

XML
<?xml version="1.0" encoding="UTF-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

   <!--Contains a set of user-defined Item elements. Every item used in a MSBuild project must be specified as a child of an ItemGroup element.-->
   <ItemGroup>
 <!--Property used to add or remove the UI component configuration or simply rules-->
 <PropertyPageSchema Include="$(MSBuildThisFileDirectory)$(MSBuildThisFileName).xml" />
 <!--The file or wildcard to include in the list of items.-->
 <AvailableItemName Include="NASM">
 <!--This makes the custom item type names available in the file property's itemtype menu-->
 <Targets>_NASM</Targets>
 </AvailableItemName>
   </ItemGroup>

   <!--A grouping element for individual properties. Properties are specified by using the Property element. They are mainly user defined properties-->
   <PropertyGroup>
 <ComputeLinkInputsTargets>$(ComputeLinkInputsTargets);ComputeNASMOutput;</ComputeLinkInputsTargets>
 <ComputeLibInputsTargets>$(ComputeLibInputsTargets);ComputeNASMOutput;</ComputeLibInputsTargets>
   </PropertyGroup>

   <!--Maps the task that is referenced in a Task element to the assembly(AssemblyName attribute) that contains the task implementation-->
   <UsingTask TaskName="NASM" TaskFactory="XamlTaskFactory" AssemblyName="Microsoft.Build.Tasks.v4.0">
 <Task>$(MSBuildThisFileDirectory)$(MSBuildThisFileName).xml</Task>
 <!--Inline Task that contains the task source code.-->
   </UsingTask>

   <!--Contains a set of tasks for MSBuild to execute sequentially.-->
   <Target Name="_NASM" BeforeTargets="$(NASMBeforeTargets)" AfterTargets="$(NASMAfterTargets)" Condition="'@(NASM)' != ''" Outputs="%(NASM.OutputFormat)" Inputs="%(NASM.Identity);%(NASM.AdditionalDependencies);$(MSBuildProjectFile)" DependsOnTargets="_SelectedFiles">
 <ItemGroup Condition="'@(SelectedFiles)' != ''">
 <NASM Remove="@(NASM)" Condition="'%(Identity)' != '@(SelectedFiles)'" />
 </ItemGroup>
 <ItemGroup>
 <NASM_tlog Include="%(NASM.OutputFormat)" Condition="'%(NASM.OutputFormat)' != '' and '%(NASM.ExcludedFromBuild)' != 'true'">
    <Source>@(NASM, '|')</Source>
 </NASM_tlog>
 </ItemGroup>

 <!--This task logs a message during a build.-->
 <Message Importance="High" Text="%(NASM.ExecutionDescription)" />
 <!--This task writes the paths of the specified items to the specified text file.-->
 <WriteLinesToFile Condition="'@(NASM_tlog)' != '' and '%(NASM_tlog.ExcludedFromBuild)' != 'true'" File="$(IntDir)$(ProjectName).write.1.tlog" Lines="^%(NASM_tlog.Source);@(NASM_tlog-&gt;'%(Fullpath)')" />
 <!--Attributes defined on the task, Its good to record all the tools parameters inside the attributes, as it will copy value from the UI during build -->
 <NASM Condition="'@(NASM)' != '' and '%(NASM.ExcludedFromBuild)' != 'true'"
   Inputs="%(NASM.Inputs)"  
   OutputFormat="%(NASM.OutputFormat)"  
   Outputswitch="%(NASM.Outputswitch)"  
   AssembledCodeListingFile="%(NASM.AssembledCodeListingFile)"  
   GenerateDebugInformation="%(NASM.GenerateDebugInformation)"  
   ErrorReporting="%(NASM.ErrorReporting)"  
   IncludePaths="%(NASM.IncludePaths)"  
   PreprocessorDefinitions="%(NASM.PreprocessorDefinitions)"  
   UndefinePreprocessorDefinitions="%(NASM.UndefinePreprocessorDefinitions)"  
   ErrorReportingFormat="%(NASM.ErrorReportingFormat)"  
   TreatWarningsAsErrors="%(NASM.TreatWarningsAsErrors)"  
   floatunderflow="%(NASM.floatunderflow)"  
   macrodefaults="%(NASM.macrodefaults)"  
   user="%(NASM.user)"  
   floatoverflow="%(NASM.floatoverflow)"  
   floatdenorm="%(NASM.floatdenorm)"  
   numberoverflow="%(NASM.numberoverflow)"  
   macroselfref="%(NASM.macroselfref)"  
   floattoolong="%(NASM.floattoolong)"  
   orphanlabels="%(NASM.orphanlabels)"  
   CommandLineTemplate="%(NASM.CommandLineTemplate)"  
   AdditionalOptions="%(NASM.AdditionalOptions)" />
   </Target>
</Project>

nasm.props:

Lastly this value can be said to be used to initialize the values. More formally, a props file contains property definitions. The very first time one clicks the property, the page appearing will set the various attributes with definition given in this file. There is not much to explain in this file and from the previous definitions all artifacts seems conspicuous.

Creation and Implementation

As the fundamental of the target file is understood, its time to create a requirement specific target file, its schema and property definition files (already shown in previous section for illustration of elements). These files are attached with the article. To make it globally available its better to copy these files (nasm.prop,nasm.targets & nasm.xml) into the MSBuild directory (for me its "C:\Program Files\MSBuild\Microsoft.Cpp\v4.0\BuildCustomizations\" )where the masm.targets exists ($(VCTargetsPath)\BuildCustomizations\).

Note: There is no need to set anything to build and run the provided sample. They are already configured to reference their own directory only.

So below is the quick summary for the configurations adopted to use build with utmost ease:-

  1. Add nasm installation path to the PATH variable or copy the executable to the %SYSTEMROOT%\System32 or whatever to make it available on command prompt with just its name.
  2. Copy nasm.prop,nasm.targets & nasm.xml to "C:\Program Files\MSBuild\Microsoft.Cpp\v4.0\BuildCustomizations\" (Explained earlier).

Now follow these steps to create a sample functional project which uses nasm.

  1. Create a C++ project (Preferably Empty Project); avoid managed projects. Name it "NASM".
  2. In the solution explorer Right Click Project name ( topmost parent node i.e. NASM ) -> Build Customizations. A pop-up will appear with nasm as an option along with masm and lc (this wont appear if you haven’t copied the target and property files in Build Customization directory, explained earlier and you have to manually locate it using "Find Existing…" option in the pop-up).Check the nasm checkbox and OK.
  3. Again in the solution explorer Right Click Source Files -> Add -> New Item -> Select C++ or header file, enter name of the file (Nasm) followed by extension .asm (i.e. Nasm.asm) and click Add. Do it once more to add Called.asm.
  4. Open the property page (through solution explorer-> right click NASM-> Properties or Select NASM and Alt+Enter or Project->Properties in menu .
  5. Select Linker in the Configuration Properties category. Expand it and click Input. In the Additional Dependencies add one more library "libcmt.lib;". You can add other as well. This is just a C Run-Time (CRT) Library.
  6. Now click Advanced option in the left pane. Yes !!!! of course , we are planning for a chaos. In this era I don’t think anyone learns assembly for printing black and white "Hello World". So set Data Execution Prevention to "No" so that we can bounce the instruction pointer to the data section.

Great, settings done, add the following code to Nasm.asm and Called.asm.

Nasm.asm

ASM
extern _printf ; the C function, to be called
extern _getchar
extern _extfun
SECTION .data ; Data section, initialized variables .: dd 5 ; int a=5;
fmt: db "a=%d, eax=%d", 10, 0 ; The printf format, "\n",'0'
msg: db "We are now executing in data area :) With DEP disabled",10,0 
Codev:
push ebp ; set up stack frame
mov ebp,esp
push dword msg ; address of ctrl string
call _printf ; Call C function
add esp, 4 ; pop stack 1 push times 4 bytes
call _extfun
mov esp, ebp ; takedown stack frame
pop ebp ; same as "leave" op
mov eax,0 ; normal, no error, return value
ret

SECTION .text ; Code section. .global _main ; the standard gcc entry point
_main: ; the program label for the entry point
push ebp ; set up stack frame
mov ebp,esp
mov eax, [a] ; put a from memory and store into register
add eax, 2 ; a+2
push eax ; value of a+2
push dword [a] ; value of variable a
push dword fmt ; address of ctrl string
call _printf ; Call C function
add esp, 12 ; pop stack 3 push times 4 bytes
mov esp, ebp ; takedown stack frame
pop ebp ; same as "leave" op 
mov eax,0 ; normal, no error, return value
call Codev ;Time to break into data section
call _getchar ;let us C ;)

ret ; return 

Called.asm

ASM
extern _printf
SECTION .data ; Data section, initialized variables .ewmsg: db "We are now from data section into an external function ;) ",10,0 


global _extfun ; the standard gcc entry point
_extfun: ; the program label for the entry point
push ebp ; set up stack frame
mov ebp,esp
push dword newmsg ; address of ctrl string
call _printf ; Call C function
add esp, 4 ; pop stack 1 push times 4 bytes
mov esp, ebp ; takedown stack frame
pop ebp ; same as "leave" op 
xor eax,eax ; normal, no error, return value
ret 

Now it’s the action time, click the green button!!. Build and run the code.Enjoy the output.

Points of Interest

Not much, but I started it in January and soon fled.

The resource available for such task is scanty and hard to search. And things worsen when your configuration makes all things a garbage. So it happened with me as well. A minor change in the target file jammed the whole process. However 5 days back I restarted working on it and my love for assembly proved something.

I was unable to test the whole configuration in the XML file, due to lack of time and in fact this is a community oriented design(In this case the NASM community), perhaps I alone am not sufficient to create it. So suggestions for modifications are always welcomed.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Systems Engineer
United States United States
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
QuestionVisual Studio's Debugger? Pin
Paul Conrad5-Sep-14 10:12
professionalPaul Conrad5-Sep-14 10:12 
QuestionFailed. It cannot linked. Pin
liaoy7476-Mar-13 22:09
liaoy7476-Mar-13 22:09 
QuestionVS2012 Pin
Michal Kowalski16-Dec-12 8:35
Michal Kowalski16-Dec-12 8:35 
SuggestionRe: VS2012 Pin
perilbrain29-Dec-12 23:44
perilbrain29-Dec-12 23:44 
AnswerRe: VS2012 Pin
Garrrett Serack30-May-13 6:59
Garrrett Serack30-May-13 6:59 
AnswerRe: VS2012 Pin
Member 363963612-Oct-13 17:51
Member 363963612-Oct-13 17:51 
QuestionIt works, But! Pin
Member 820224422-Aug-12 2:56
Member 820224422-Aug-12 2:56 
AnswerRe: It works, But! Pin
perilbrain22-Aug-12 6:16
perilbrain22-Aug-12 6:16 
AnswerRe: It works, But! Pin
Ashod20-Sep-12 6:27
Ashod20-Sep-12 6:27 
AnswerRe: It works, But! Pin
ghostmonk22-Sep-12 4:29
ghostmonk22-Sep-12 4:29 
AnswerRe: It works, But! Pin
Senta16-Oct-12 6:25
Senta16-Oct-12 6:25 
QuestionWhat about GCC Pin
douglasmota3-Jul-12 7:17
douglasmota3-Jul-12 7:17 
AnswerRe: What about GCC Pin
perilbrain4-Jul-12 23:12
perilbrain4-Jul-12 23:12 
AnswerRe: What about GCC Pin
douglasmota5-Jul-12 1:17
douglasmota5-Jul-12 1:17 
GeneralMy vote of 5 Pin
JF20152-Jul-12 0:31
JF20152-Jul-12 0:31 
GeneralRe: My vote of 5 Pin
perilbrain2-Jul-12 0:55
perilbrain2-Jul-12 0:55 
GeneralMy vote of 5 Pin
NarangGopal29-Jun-12 9:09
NarangGopal29-Jun-12 9:09 
QuestionNeeds some work Pin
Richard MacCutchan26-Jun-12 22:46
mveRichard MacCutchan26-Jun-12 22:46 
GeneralRe: Needs some work Pin
perilbrain26-Jun-12 22:58
perilbrain26-Jun-12 22:58 
GeneralRe: Needs some work Pin
Richard MacCutchan26-Jun-12 23:13
mveRichard MacCutchan26-Jun-12 23:13 
GeneralRe: Needs some work Pin
perilbrain26-Jun-12 23:53
perilbrain26-Jun-12 23:53 
Thank you very much I'll ask them.........

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.