Setup for Test Servers to Run Coded UI Tests from VSO/TFS 2015

Introduction

In order to run Coded UI Tests on Visual Studio Online or TFS 2015 On Premise, the test run servers need to be setup with several things which are not intuitive or setup automatically. There may be some extra steps needed depending on your system but this is the minimum I have had to do to get my servers up and running.

There are lots of tutorials and guides for setting up builds or releases to run tests, but I have never seen anything on the specific setups for the servers themselves to get them ready to and be ready to run tests.

I am not going to go into specific detail on some items, but this should at least get people around some of the issues and give some pointers in the right direction.

The Setup

• Download and Install the 2015 Test Agent. Download from here.
• Setup Windows Remote Management - Open a command prompt as admin and run the command "winrm quickconfig". If you do not do this, you will get ambiguous errors about not connecting to the server.
• Disable lock screen
  • Run > gpedit.msc > Administrative Templates/Control Panel/Personalization/Do not display the lock screen > enabled
• Enable auto login- Run > Regedit > HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows NT/CurrentVersion/Winlogon >
  • Edit AutoAdminLogin to data = 1
  • Edit DefaultDomain to your domain (if you are on a domain)
  • Edit DefaultUserName to be the user to login that will run tests
  • Add/Edit DefaultPassword set to users password (if the value does not exist, add it by right clicking WinLogon and choosing New > "String Value"
• Restart the computer.

Points of Interest

• When the first test run starts, the test run server will restart and auto login as the user and work, even if the user was already auto logged in. But also as long as you do not remote into the computer or change it in any way, subsequent logins will no longer restart that computer. If you do not disable the lock screen, it will restart every time as it will be locked.
• Using Hyper-V, there is a small view of the desktop of the computer so you can tell what state the computer is in, i.e., logged in or running tests or restarting, etc.
• Reinstalling any version of a test agent will remove registry setting for Default Password, I have not seen it modify the auto admin login setting, the domain or the default user. Also, the group policy lock does not seem to be effected. I am unsure how this will be effected if you let the Deploy Test Agent Task install the agent.
• I manually install the agents on the server and uncheck Update Test Agent. This stops the system from downloading the agent and saves time if you have a slow network.

History

Changed information for setting up DefaultPassword. Previously said "add new key", changed to "New > "String Value"". The entry is a String Value, not a Key.