IDF2015 Lab Notes: Getting Your Nexus Player from Shrink-Wrap to Performance and Energy Analysis

Intel® Developer Zone offers tools and how-to information for cross-platform app development, platform and technology information, code samples, and peer expertise to help developers innovate and succeed. Join our communities for Android, Internet of Things, Intel® RealSense™ Technology, and Windows to download tools, access dev kits, share ideas with like-minded developers, and participate in hackathon’s, contests, roadshows, and local events.

Introduction

Initially presented as a hands-on lab for Intel Developer Forum 2015 as "SFTL002: Hands-on Lab: Intel® Architecture Based Mobile Device Development Workshop – Featuring Android* TV" the lab instructions are provided here as reference in the hope that they are useful to a wider developer community. The boot image and appropriate data capture drivers were built prior to the lab by the presenter but notes are included on how to perform both tasks in the notes, although not in as great a detail. Regardless I hope that the following information will be useful to you, all tasks are possible with a commercially available Nexus Player device, a host PC running Ubuntu, a properly licensed version of Intel System Studio, and internet access to publicly available referenced sites. Versions used are as follows: Nexus Player: Build LMY47V but initial build on device often much earlier, Ubuntu 14.04 LTS, ISS 2016-Beta with SoC Watch 2.0-Beta.

Table of Contents

1. Basic Familiarization with the Nexus Player
2. Recovering to Factory Image
3. Rooting the Device
4. Running a Performance Analysis
5. Reviewing the Results
6. Installing Additional Drivers
7. Re-Running Analysis and Comparing with prior results
8. Running Energy Profile and Power Analysis
9. Building the boot.img needed to Root the Device
10. Building the Additional Drivers
11. Notes for setting up Ubuntu on the host

Booting your Nexus Player and getting familiar with it

• Batteries in the remote

• Plug in the USB cable from your device to your PC, and the HDMI cable from your device to your display, THEN plug in the device power cable.

• First thing the device will request is to sync with your remote, this is done by holding down the left arrow and circle key on the remote as indicated on the device display.

• Then select your language and WiFi network for the device by using the remote

  • The player needs non-proxy access to the internet through a WiFi connection to continue. Some basic security setups are supported but not a username/password combination system.

• Then sign in to a google account, you may sign in with your own account a dummy gmail account was created for use in the lab

• You may view Google terms of service or select continue

• I recommend not sharing location, but that is up to you.

• On your PC open a command prompt and check lsusb and adb devices. You will notice that the device is not listed.

• Select Settings and "About" then scroll down to "Build" and click on the selector until you receive the message "You are now a Developer" Take a note of the build version. You will need to build your boot.img (in this step) to match the build version on your device, or flash the device to a factory default image matching the build version desired (as in this step). More on this later.

• Now back out of the settings menu and return once again, you should now be able to select "Developer options" under the "Preferences" section

• Under "Debugging" set "USB debugging" to ON, this will result in a prompt requesting access from you PC system.Check the "always allow this device" and then select OK.

• On your PC re-run lsusb and adb devices and you will now be able to see the device

• You may now side-load some applications if desired and/or copy over some movie files for viewing on the device. In the lab some public domain NASA videos were used as was the VLC player and EC File Manager app otherwise available on the Google Play store.

  • adb push <filename>.mp4 /sdcard/Movies

  • adb install <filename>.apk (device will prompt for verification)

• Note that you may or may not be able to see the location where you copied the video files as the device is still not rooted.

  • adb shell

    • ls /sdcard

    • ls /sdcard/Movies

• Yet they clearly exist and can be used (you can’t see the Movies directory, but you can see what files you just put there if you know the directory exists)

  • On device us File manager (just installed) to navigate and find files

  • On device use VLC player to play a video clip

• Now we are familiar with the device and the some of the limitations of having a non-rooted device as well as some of the ways to get around this without rooting the device

Recovery to factory settings

• Factory recovery images for the Nexus Player are available directly from Google via this site: https://developers.google.com/android/nexus/images

• Boot the Nexus Player (device) by plugging it in, ensure HDMI cable connects to monitor, USB connects to laptop, and power connects to power

• On you Host machine open a terminal and navigate to the directory where you unzipped the downloaded image

• Confirm connection with device by running adb devices

• Boot into the bootloader via adb reboot bootloader

  • This may also be accomplished by holding down the button on the bottom of the device during initial boot.

• Disable the oem lock to enable flashing with fastboot oem unlock. If this fails don’t be concerned just try again, sometimes it takes a couple of attempts.

• Revert device to factory image LMY47V by running ./flash-all.sh

  • Device will reboot at least once after flashing the bootloader itself, after which it will flash the rest of the system.One may follow the progress both on one’s system as well as one’s device display.Upon completion the device will reboot once again and should come up with factory image as though it were brand new out of the box!

Rooting the device

• If you attempted to run VTune prior to rooting the device you would get an error message when it attempts to install the necessary app code onto the device itself. Although VTune will often run and provide useful data even on un-rooted devices my experience with this particular device while preparing for this lab was that it did not. Thus we root the device first.

• On the PC navigate to the directory containing your boot.img as built here

• Run adb reboot bootloader and wait for your device to reboot into the bootloader screen

• Run fastboot oem unlock when successful the oem lock state should be updated on the device screen.If this fails don’t be concerned just try again, sometimes it takes a couple of attempts

• The boot.img we have prepared for this lab is based on lollipop build LM47V which is why we needed to update the device to that build version in order for this step to work. Should a different build need to be rooted then one would simply need to create a boot.img file for the appropriate version. This is because we are only flashing the boot.img file and it will need to match the build version of the system.img and other components of the device which are not updated with this step.

• Run fastboot flash boot boot.img if that completes without error then continue, otherwise retry, it should only take a second or two

• At this point one could choose to re-lock the oem lock via fastboot oem lock, but it’s not necessary.

• Run fastboot continue to reboot the device back to normal user interface

• You should note on the boot screen now displays "Google" with a little unlocked lock symbol, previously there was no lock symbol on this screen. It will then continue booting back to the normal UI.

Running a performance analysis

• Run adb root to confirm that the rooting worked and gain root access through adb

• You may now adb shell to the device and see what directory structure is visible which previously wasn’t (such as /data)

• Open VTune Amplifier on the host system

  • Run /opt/intel/vtune_amplifier_for_system/bin64/amplxe-gui

  • In the "Welcome" screen select "New Project...", or use the icon in the menu bar at the top, or the drop-down menu selection

  • Input a project name such as "IDF_DemoLab", leave the project location as default

  • On the left of the screen (ignoring the Project Navigator which you may close if you wish) select the Analysis Target "Android device (ADB)"

  • You may receive a "Verify apps?" prompt on the device at this point, go ahead and Accept.

  • Default values of "Launch Application" via ADB on device "######" should be visible (in other words it should auto detect the existence of the Nexus Player at this point).Change "Launch Application" to "Launch Android Package"

  • Select the Package name either by typing it in or by browsing for the VLC player previously installed. "org.videolan.vlc"

  • We’ll limit the length of data capture for the purposes of this lab

    • Check the "Automatically stop collection after (sec):" and put in a value of 30.This will automatically stop data collection after 30 seconds.

    • Leave the box for "Automatically resume collection after (sec):" unchecked and value blank.Note that using this option would allow for a specified number of seconds after the app launches for you to select a video to play before collection starts.For this lab go ahead and leave it blank and unchecked so we’ll make sure to analyze user activities as well as the automatic operations.

  • All other default selections should be fine, but feel free to explore and become a bit more familiar with what can be changed.

  • Select "Choose Analysis" or select the "Analysis Type" tab to get to the next step.

    • Note: "Binary/Symbol Search" and "Source Search" buttons below the "Choose Analysis" are used to provide VTune with Symbol and Source code information for more meaningful analysis results when profiling your own code. The directory information for these files can be provided now or later simply by accessing the project properties or results screen and re-resolving symbol information (another button).

• Now that the device is rooted you’ll notice we’ve got the option for several types of analysis such as "Basic Hotspots", "Advanced Hotspots", and "General Exploration"

• For this example we’ll pick "Advanced Hotspots"

  • Use the default value of 1 for the CPU sampling interval

  • Select the "Hotspots" level of detail.Note some of the warning messages which appear when attempting other levels of detail as greater level of driver support enables greater levels of data collection.

  • Check the "Analyze user tasks" selection box

  • Uncheck the "Analyze GPU usage" as we don’t have Intel HD Graphics on this device, and ensure the "Analyze Processor Graphics hardware events:" selection is set to "None"

  • When ready go ahead and click "Start"

• If the screen-saver on your device has triggered then click the remote to go back to the main menu, you should notice a "Waiting for debugger" message and then VLC will launch.

• Go ahead and select a video to play during the collection (recommend the "Earth Timelapse")

• After 30 seconds VLC will automatically close and the results will be pulled off the device and into VTune

• At this point you will see a lot of warnings scrolling past in the collection log screen.This is because we have not specified where VTune can find the symbol information for all the function calls associated with every piece of code run during the collected event.

Reviewing the results

• At this point we will take just a brief moment to look at the results being shown in VTune, starting with the summary screen which immediately pops up upon completion of processing the collected data.

• Hotspots Viewpoint (with pop-up description and option to de-select showing this pop-up every time)

  • Note that clicking on "change" will allow different viewpoints to be selected.This is still the same data from the same collected event, however the analysis is presented using slightly different metrics.This allows you to pick a viewpoint most appropriate to your situation.Here we will stick with Hotspots Viewpoint.

• Summary tab: Basic high level information, how long the collection took, top most active functions during the event, CPU Usage histogram, collection and platform information.Note that on the CPU Usage Histogram you may use the sliders to re-define "poor", "ok", and "ideal".

• Bottom-up tab: Easy way to quickly identify which specific function is taking most of the time during the collection event. Particularly useful if you’ve got a code bottleneck causing a problem.

  • Several options available in the "Grouping" to change how the CPU time Pareto data can be grouped for display

  • Graph on the bottom can similarly be changed whether to show data by thread, process, etc.

  • Graph may also be used to zoom in, highlight and select specific time slice, even filter other data to a specific time segment so that you can then view the above Pareto for information filtered around a specific time of interest.

• < >

  Top-Down Tree: Similar to the bottom-up display but instead looking at the function stack from the top down.It’s worth noting that double clicking on a specific function name either here or in the bottom-up view will open a new tab showing the specific code segment of interest.If proper symbol and source data is provided then this will take you to the specific source code file, otherwise the assembly code is showed.

• Platform: basically a full screen view of the time-plot shown at the bottom of the Bottom-up and Top-Down screens. Allows for much better visibility. Notice that hovering the mouse pointer over various items will result in a pop-up text with some additional information such as thread ID, CPU utilization or similar

Installing additional drivers

• Here we will install some additional drivers to improve collection capabilities beyond what’s by default built into the kernel.In this case we’re adding modular drivers which can be installed and removed while the device is running.Optionally when developing a device image yourself you can build these drivers in directly (typically done for the sep and pax drivers).

• Ensure that you’ve got root access on the device by running adb root

• Shell over to the device with adb shell

• Re-mount the root filesystem as read-write (you may confirm prior to this that it’s read only by trying to create a new directory) with mount –o rw,remount /

• Create a /lib directory with mkdir /lib then a modules subdirectory with mkdir /lib/modules

• Note that there is already a /system/lib directory and normally we would have a modules subdirectory there in which to put our drivers.Unfortunately experimentation with this device showed insufficient disk space in that directory structure to copy over the files, so we go with the alternate location /lib/modules.VTune automatically looks for several of these drivers in both /system/lib/modules and /lib/modules as their location can vary from device to device

• Exit from your adb shell and then push the ko driver files from their directory on your host

  • < >

    adb push pax.ko /lib/modules

  • adb push sep3_15.ko /lib/modules

  • adb push socperf1_2.ko /lib/modules

  • adb push socwatch2_0.ko /lib/modules

• Go ahead and load the pax and sep drivers on the device before running another Hostspot analysis.

  • adb shell

  • insmod /lib/modules/pax.ko

  • insmod /lib/modules/sep3_15.ko

  • < >

    Now you can verify that they were used by checking the ~/intel/amplsys/projects/<your project name>/<collection run>/data.0/ directory. Note that your first run (likely the r000ah run directory) indicates usage of the perf driver for collection as you can see from some of the filenames. Your next run should have some sep######.tb6 file.

  • Take another event collection just like in step 3 but while the collection is happening try doing something slightly different, such as jump around to different points in the video using the remote

  • Now you may take some time like in section 4 to review the results, notice that in addition to the "Welcome" tab your results show up in different tabs near the top of the screen with names for your results such as "r001hs" and "r002hs".It is possible to switch back and forth between these tabs and look at the differences between these events, but that can become difficult.

  • Select "Compare Results" using either the icon which looks like a circle cut into two half circles byvertical stripe, or using the keyboard short-cut CTRL+ALT+O

  • Select your two runs of interest using the pull down menus for Result 1 and Result 2, then click on "Compare"

  • This may require closing the tabs of the specific results in some versions of VTune (but not in this one), and will open a new tab labeled something like "r001hs-r002hs" with some direct metric by metric comparison information shown for the two results as well as the difference.

  • Explore around, imagine how this might be useful for your specific application

• Power/Energy profiling using Intel Energy Profiler

  Building the root boot.img image

  Setup build environment, and perform repo sync per instructions at:

  http://source.android.com/source/building.html

  While doing the repo init make sure to specify the appropriate branch as determined by the lookup table here: http://source.android.com/source/build-numbers.html#source-code-tags-and-builds

  Note: the IDF lab was done using branch android-5.1.1_r1 for build LMY47V, at the time the latest build available for the Nexus Player device

  After initial AOSP repo sync is finished but before running make:

  Checkout kernel source:

  git clone https://android.googlesource.com/kernel/x86_64.git

  cd x86_64

  git branch –a (see a list of branches available)

  git checkout android-x86_64-fugu-3.10-lollipop-mr1

  make fugu_defconfig

  NOW EDIT THE .config – Where CONFIG_MODULES is not set, change to CONFIG_MODULES=y

  (There will be additional config options needing answers – I don’t recall answering YES to any of the additional prompts, but memory may be a fickle thing.)

  Make –j8

  cd ..

  Download all the proprietary stuff (mentioned in the source.android.com instructions under "Building for Devices" and located here https://developers.google.com/android/nexus/drivers ):

  wget https://dl.google.com/dl/android/aosp/asus-fugu-lmy47v-f7524060.tgz

  wget https://dl.google.com/dl/android/aosp/broadcom-fugu-lmy47v-8ce8ebda.tgz

  wget https://dl.google.com/dl/android/aosp/google-fugu-lmy47v-c77f7094.tgz

  wget https://dl.google.com/dl/android/aosp/intel-fugu-lmy47v-0bb60afb.tgz

  wget https://dl.google.com/dl/android/aosp/widevine-fugu-lmy47v-ee5d30ed.tgz

  NOW execute the ‘extract-*.sh’ scripts which came from the tarballs

  Build the bootimage:

  source build/envsetup.sh

  lunch aosp_fugu-userdebug

  export TARGET_PREBUILT_KERNEL=<AOSP_PATH>/x86_64/arch/x86/boot/bzImage

  make bootimage –j8

  boot.img should be located in <AOSP_PATH>/out/target/product/fugu/

   

  Building the SEP and SoC Watch drivers

  SoC Watch driver build:

  Download the available and desired version of SoC Watch (included in your distribution of Intel System Studio in the targets/system_studio_target.tgz zip file).

  Follow the instructions in the included User Guide pdf document (usually section 2.2, "Building the Kernel Modules").

  GOTCHA#1: Make sure to specify the kernel-build-directory correctly when prompted while building the soc_perf_driver. Kernel-build-directory will be the <AOSP_SRC>/x86_64/ directory you created while building the kernel.

  GOTCHA#2: When building the socwatch_driver make sure to include the –s option with the path to the Module.symvers file in the soc_perf_driver/src/ directory created during the soc_perf_driver build step.

  SEP driver build:

  Very similar to the SoC Watch driver build. Necessary files are included in the VTune Amplifier component of Intel System Studio in the /opt/intel/vtune_amplifier_for_systems/sepdk directory. Simply cd into that directory and run the ./build-driver script, making sure to provide the appropriate path to the kernel-build-directory in order to get the security signature correct. This should create all the drivers needed, however it may be necessary to go into the pax and vtsspp subdirectories and run their respective build-driver scripts to ensure that those drivers get built. The first should create a sepx_xx.ko where the x’s are replaced by the specific version of the driver (such as sep3_15.ko from our lab example), the pax directory should create a pax.ko file.

  • Energy Analysis with the socwatch collector is a much more manual process involving running the collector manually on the device through an adb shell, or through the use of a specially designed app to run the collector

  • Install the socwatch collector onto the device from the host. Navigate to the SoC Watch directory and run the install script ./socwatch_android_install.sh

    • Note: SoCWatch collector can be obtained from your installation of Intel System Studio by unzipping the system_studio_target.tgz file located in the Targets sub-directory of the Intel System Studio install directory. This unzipped directory will contain both the referenced install script (for both Windows and Linux) as well as a User Guide in pdf format.

  • Shell to the device, setup the run environment and load the driver modules

    • adb root

    • adb shell

    • cd /data/socwatch

    • . ./setup_socwatch_env.sh (you may also run source ./setup_socwatch_env.sh as the idea is to apply all of the script environment settings to the current shell)

    • insmod /lib/modules/socperf1_2.ko

    • insmod /lib/modules/socwatch2_0.ko

  • Now get ready to run a collection, navigate as desired using the device remote depending upon what you wish to profile (get ready to launch your app, or leave it in screen saver if you’re profiling the device as idle)

  • Now run a socwatch collection, you may look at several of the possible options with the socwatch –h command.Here is an example which will run a collection for 30 seconds and save the results in the /data/socwatch/results/ directory.

    • Note that each run you will want to use a different name for the results, to avoid overwriting previous results and possibly adding some confusion with multiple results existing in the same file.

    • socwatch –m –f cpu –f sys –t 30 –r vtune –o ./results/test1

    • Now check the results directory and you should find three files, test1.csv test1.sw2 and test2.pwr.The csv file is a comma separated value file for use in third party analyzers, the pwr file is the input file for VTune, and the sw2 is the raw output file.

      • Note: If you were running socwatch 1.x then there would be no pwr file and you would directly import the sw1 file into VTune.

  • Exit from the device shell, and pull the results into your SWResults directory

    • < >

      cd ~/IDFLab/SWResults

    • adb pull /data/socwatch/results/test1.pwr

  • Load the results in VTune.

    • In VTune select "Import Results" on the Welcome tab or the corresponding icon at the top of the VTune window, or use the keyboard shortcut CTRL+ALT+N

    • Under "Import a directory" select browse and select the ~/IDFLAB/SWResults/ directory with the file you just pulled from the device.Then click Import.

      • Note: This was only necessary due to the specific pre-production versions of software we are using for this lab.For any version you will have which includes a copy of SoCWatch it will be able to import the single individual file with the "import a single file" option and then select the specific sw1 or pwr file of interest (sw1 files are imported from SoCWatch version 1.x, pwr files are imported from SoCWatch 2.0)

  • The results will be processed and then displayed much in the same way the performance analysis is displayed.Notice you still have the Summary tab, but you now have CPU C/P States, core Wake-ups, Correlate Metrics, and the selected viewpoint is "Platform Power Analysis viewpoint" rather than "Hotspots viewpoint".Histogram information on the summary screen now shows CPU frequency and Core sleep state rather than showing CPU Usage.

    • Note: More tabs with more data will be available depending upon the specific –f feature options used during the collection.

Notes for setting up Ubuntu to work for the lab host system

Create a file: /lib/udev/rules.d/51-android.rules this will allow adb to access the device

• SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", MODE="0666"
  

  apt-get install android-tools-adb android-tools-fastboot

  Or NOT, having some trouble with android-tools-fastboot. Seems to work when using SDK/platform-tools/fastboot and adb as provided by the Android SDK, so may stick with that.

  JDK:

  Apt-get install openjdk-7-jdk

  Or

  Apt-add-repository ppa:webupd8team/java

  Apt-get update

  Apt-get install oracle-jdk7-installer

  Need ia32-libs

  Apt-get install lib32z1

  ~/.bashrc edit

  #AndroidDev PATH 
  
  export PATH=$PATH:/home/username/sdk/tools

  Legal Stuff

  Intel, VTune, and the Intel logo are trademarks of Intel Corporation in the United States and other countries.

  *Other names and brands may be claimed as the property of others.

  © 2015 Intel Corporation