2012年6月6日 星期三

Tegra 3 Matrix Mul Performance Test

今天以矩陣相乘為例,測試不同的編譯參數對效能的影響。有些編譯參數應該是沒有直接fit 最佳化的狀況因此最佳化結果不明顯。但這只是初期的測試,未來要更深入的研究一下每一個編譯參數實際上是對什麼case做最佳化。 

測試的數據是以arm-eabi-g++來編譯,後來有試過改成可以讓arm-eabi-gcc來編譯的版本,測試結果其實差不多。以下是測試的程式及環境


  • 測試程式
      #include <sys/time.h>
      #include <math.h>
      #include <stdio.h>
      #include <stdlib.h>
      
      using namespace std;
      
      long double timer[10];
      int matrix_size;
      /*Timer*/
      long double high_prec_time(){
              timeval tv;
              gettimeofday(&tv,0);
              return (long double)tv.tv_sec+tv.tv_usec*1e-6;
      }
      void   set_timer(int k=0){
              timer[k]=high_prec_time();
      }
      double get_timer(int k=0){
              return (double)(high_prec_time()-timer[k]);
      }
      
      /*Matrix Mul*/
      template <class T>
      void _matrix_mul(T *matrixA,T *matrixC){
           T tmpA;
          for(int i=0;i<matrix_size;i++){
              for(int j=0;j<matrix_size;j++){
                  for(int k=0;k<matrix_size;k++){
                      tmpA=*(matrixA+k*matrix_size+j);
                      //cout<<tmpA<<endl;
                      *(matrixC+(i*matrix_size+j)) += tmpA*tmpA;
                  }
              }
          }
      }
      void int_matrix_mul(int matrix_size){
          int matrixA[matrix_size][matrix_size];
          int matrixC[matrix_size][matrix_size];
          //srand(time(0));
          for(int i=0;i<matrix_size;i++){
              for(int j=0;j<matrix_size;j++){
                  matrixA[i][j]=rand()%RAND_MAX;
                  //cout<<matrixA[i][j]<<RAND_MAX<<endl;
                  matrixC[i][j]=0;
              }
          }
          _matrix_mul(&matrixA[0][0],&matrixC[0][0]);
      }
      
      void float_matrix_mul(int matrix_size){
          float matrixA[matrix_size][matrix_size];
          float matrixC[matrix_size][matrix_size];
          //srand(time(0));
          for(int i=0;i<matrix_size;i++){
              for(int j=0;j<matrix_size;j++){
                  matrixA[i][j]=(float)(rand()%RAND_MAX);
                  matrixC[i][j]=0;
              }
          }
          _matrix_mul(&matrixA[0][0],&matrixC[0][0]);
      }
      void double_matrix_mul(int matrix_size){
          double matrixA[matrix_size][matrix_size];
          double matrixC[matrix_size][matrix_size];
          //srand(time(0));
          for(int i=0;i<matrix_size;i++){
              for(int j=0;j<matrix_size;j++){
                  matrixA[i][j]=(double)(rand()%RAND_MAX);
                  matrixC[i][j]=0;
              }
          }
          _matrix_mul(&matrixA[0][0],&matrixC[0][0]);
      }
      
      int main(){
          matrix_size=512;
          set_timer(0);
          int_matrix_mul(matrix_size);
          printf("Integer Matrix Mul spend time: %f",get_timer(0));
          set_timer(0);
          float_matrix_mul(matrix_size);
          printf("Float Matrix Mul spend time: %f",get_timer(0));
          set_timer(0);
          double_matrix_mul(matrix_size);
          printf("Double Matrix Mul spend time: %f",get_timer(0));
      
      }
      
  • 測試環境
      硬體: 
              SOC: TEGRA3
              RAM: 1G 
      軟體:
              ICS 4.0.4
              GCC: arm-eabi-g++ 4.4.3

    測試的方式是每一個編譯出來的執行檔會測試四次,去掉特別大或特別小的值取出三個。測試分別對Integer, Float及 Double的size為256 的square matrix測試。下圖則是測試的結果。

Compiler 做了哪些最佳化?

我們都知道在編譯的時候可以下 O1 O2 等等的最佳化設定,但這些設定到底實際上做了哪些最佳化呢?我們可以利用下面這個指令知道:

$ arm-eabi-gcc -c -Q -O3 --help=optimizers 

以上面這個例子來說,當我們下-O3的時候實際上做了:

The following options control optimizations:
  -O                    
  -Os                           
  -falign-functions             [enabled]
  -falign-jumps                 [enabled]
  -falign-labels                [enabled]
  -falign-loops                 [disabled]
  -fargument-alias              [enabled]
  -fargument-noalias            [disabled]
  -fargument-noalias-anything   [disabled]
  -fargument-noalias-global     [disabled]
  -fasynchronous-unwind-tables   [disabled]
  -fbranch-count-reg            [enabled]
  -fbranch-probabilities        [disabled]
  -fbranch-target-load-optimize  [disabled]
  -fbranch-target-load-optimize2  [disabled]
  -fbtr-bb-exclusive            [disabled]
  -fcaller-saves                [enabled]
  -fcommon                      [enabled]
  -fconserve-stack              [disabled]
  -fcprop-registers             [enabled]
  -fcrossjumping                [enabled]
  -fcse-follow-jumps            [enabled]
  -fcse-skip-blocks             [disabled]
  -fcx-fortran-rules            [disabled]
  -fcx-limited-range            [disabled]
  -fdata-sections               [disabled]
  -fdce                         [enabled]
  -fdefer-pop                   [enabled]
  -fdelayed-branch              [disabled]
  -fdelete-null-pointer-checks   [enabled]
  -fdominance-check             [disabled]
  -fdse                         [enabled]
  -fearly-inlining              [enabled]
  -fexceptions                  [disabled]
  -fexpensive-optimizations     [enabled]
  -ffinite-math-only            [disabled]
  -ffloat-store                 [disabled]
  -fforward-propagate           [enabled]
  -fgcse                        [enabled]
  -fgcse-after-reload           [enabled]
  -fgcse-las                    [disabled]
  -fgcse-lm                     [enabled]
  -fgcse-sm                     [disabled]
  -fgraphite-identity           [disabled]
  -fguess-branch-probability    [enabled]
  -fhandle-exceptions           
  -fif-conversion               [enabled]
  -fif-conversion2              [enabled]
  -finline-functions            [enabled]
  -finline-functions-called-once  [enabled]
  -finline-small-functions      [enabled]
  -fipa-cp                      [enabled]
  -fipa-cp-clone                [enabled]
  -fipa-matrix-reorg            [disabled]
  -fipa-pta                     [disabled]
  -fipa-pure-const              [enabled]
  -fipa-reference               [enabled]
  -fipa-type-escape             [disabled]
  -fivopts                      [enabled]
  -fjump-tables                 [enabled]
  -flimit-hot-components        [disabled]
  -floop-block                  [disabled]
  -floop-interchange            [disabled]
  -floop-strip-mine             [disabled]
  -fmath-errno                  [enabled]
  -fmerge-all-constants         [disabled]
  -fmerge-constants             [enabled]
  -fmodulo-sched                [disabled]
  -fmove-loop-invariants        [enabled]
  -fnon-call-exceptions         [disabled]
  -fomit-frame-pointer          [enabled]
  -foptimize-register-move      [enabled]
  -foptimize-sibling-calls      [enabled]
  -fpack-struct                 [disabled]
  -fpack-struct=        
  -fpeel-loops                  [disabled]
  -fpeephole                    [enabled]
  -fpeephole2                   [enabled]
  -fpessimistic-inline-stack-limit  [enabled]
  -fpredictive-commoning        [enabled]
  -fprefetch-loop-arrays        [disabled]
  -fprofile-dump                [disabled]
  -frecord-options-in-elf       [disabled]
  -freg-struct-return           [enabled]
  -fregmove                     [enabled]
  -frename-registers            [enabled]
  -freorder-blocks              [enabled]
  -freorder-blocks-and-partition  [disabled]
  -freorder-functions           [enabled]
  -frerun-cse-after-loop        [enabled]
  -freschedule-modulo-scheduled-loops  [disabled]
  -frounding-math               [disabled]
  -frtti                        
  -fsample-profile              [disabled]
  -fsample-profile-use-entry    [disabled]
  -fsched-interblock            [enabled]
  -fsched-spec                  [enabled]
  -fsched-spec-load             [disabled]
  -fsched-spec-load-dangerous   [disabled]
  -fsched-stalled-insns         [disabled]
  -fsched-stalled-insns-dep     [enabled]
  -fsched2-use-superblocks      [disabled]
  -fsched2-use-traces           [disabled]
  -fschedule-insns              [enabled]
  -fschedule-insns2             [enabled]
  -fsection-anchors             [enabled]
  -fsel-sched-pipelining        [disabled]
  -fsel-sched-pipelining-outer-loops  [disabled]
  -fsel-sched-reschedule-pipelined  [disabled]
  -fselective-scheduling        [disabled]
  -fselective-scheduling2       [disabled]
  -fshort-double                
  -fshort-enums                 
  -fshort-wchar                 
  -fsignaling-nans              [disabled]
  -fsigned-zeros                [enabled]
  -fsingle-precision-constant   [disabled]
  -fsplit-ivs-in-unroller       [enabled]
  -fsplit-wide-types            [enabled]
  -fstrict-aliasing             [enabled]
  -fstrict-enum-precision       [enabled]
  -fthread-jumps                [enabled]
  -fno-threadsafe-statics       
  -ftoplevel-reorder            [enabled]
  -ftrapping-math               [enabled]
  -ftrapv                       [disabled]
  -ftree-builtin-call-dce       [enabled]
  -ftree-ccp                    [enabled]
  -ftree-ch                     [enabled]
  -ftree-copy-prop              [enabled]
  -ftree-copyrename             [enabled]
  -ftree-cselim                 [enabled]
  -ftree-dce                    [enabled]
  -ftree-dominator-opts         [enabled]
  -ftree-dse                    [enabled]
  -ftree-fre                    [enabled]
  -ftree-loop-distribution      [disabled]
  -ftree-loop-im                [enabled]
  -ftree-loop-ivcanon           [enabled]
  -ftree-loop-linear            [disabled]
  -ftree-loop-optimize          [enabled]
  -ftree-lr-shrinking           [enabled]
  -ftree-lrs                    [disabled]
  -ftree-pre                    [enabled]
  -ftree-reassoc                [enabled]
  -ftree-scev-cprop             [enabled]
  -ftree-sink                   [enabled]
  -ftree-sra                    [enabled]
  -ftree-switch-conversion      [enabled]
  -ftree-ter                    [enabled]
  -ftree-vect-loop-version      [enabled]
  -ftree-vectorize              [enabled]
  -ftree-vrp                    [enabled]
  -funit-at-a-time              [enabled]
  -funroll-all-loops            [disabled]
  -funroll-loops                [disabled]
  -funsafe-loop-optimizations   [disabled]
  -funsafe-math-optimizations   [disabled]
  -funswitch-loops              [enabled]
  -funwind-tables               [disabled]
  -fvar-tracking                [enabled]
  -fvar-tracking-uninit         [disabled]
  -fvariable-expansion-in-unroller  [disabled]
  -fvect-cost-model             [disabled]
  -fvpt                         [disabled]
  -fweb                         [enabled]
  -fwhole-program               [disabled]
  -fwrapv                       [disabled]

Cross Compile C/C++ using arm-eabi-gcc/g++( provide by Android) Directly

上一篇有提到我們可以使用agcc來幫我們解決Include header及Linking Library的問題。但因為他已經預設加入一些額外的編譯參數例如-march=armv5te等等,要一個個修改也頗煩不如直接寫個別project寫一個Makefile。因此參考agcc的方式下面show出我的Makefile的code。
P.S. 如同上一篇,在使用前必須要先Build Android來產生相關的Library

CPP = arm-eabi-g++
CC = arm-eabi-gcc
PRODUCT = cardhu
Android_PATH = /opt/compal/android/15r1
NAME = matrix_mal
CFLAG = -I$(Android_PATH)/system/core/include \
        -I$(Android_PATH)/hardware/libhardware/include \
        -I$(Android_PATH)/hardware/ril/include \
        -I$(Android_PATH)/dalvik/libnativehelper/include\
        -I$(Android_PATH)/frameworks/base/include \
        -I$(Android_PATH)/external/skia/include \
        -I$(Android_PATH)/out/target/product/$(PRODUCT)/obj/include\
        -I$(Android_PATH)/bionic/libc/arch-arm/include\
        -I$(Android_PATH)/bionic/libc/include \
        -I$(Android_PATH)/bionic/libstdc++/include\
        -I$(Android_PATH)/bionic/libc/kernel/common\
        -I$(Android_PATH)/bionic/libc/kernel/arch-arm\
        -I$(Android_PATH)/bionic/libm/include \
        -I$(Android_PATH)/bionic/libm/include/arch/arm \
        -I$(Android_PATH)/bionic/libthread_db/include\
        -I$(Android_PATH)/bionic/libm/arm\
        -I$(Android_PATH)/bionic/libm\
        -I$(Android_PATH)/out/target/product/$(PRODUCT)/obj/SHARED_LIBRARIES/libm_intermediates\
        -include $(Android_PATH)/system/core/include/arch/linux-arm/AndroidConfig.h

LDFLAGS = -mthumb-interwork -fpic -fno-exceptions -ffunction-sections \
        -funwind-tables -fstack-protector\
         -fmessage-length=0 -Bdynamic -Wl,-T,$(Android_PATH)/build/core/armelf.x\
         -Wl,-dynamic-linker,/system/bin/linker -Wl,--gc-sections -Wl,-z,nocopyreloc \
         -Wl,--no-undefined -Wl,-rpath-link=$(Android_PATH)/out/target/product/$(PRODUCT)/obj/lib \
         -L$(Android_PATH)/out/target/product/$(PRODUCT)/obj/lib \
         -nostdlib $(Android_PATH)/out/target/product/$(PRODUCT)/obj/lib/crtend_android.o \
         $(Android_PATH)/out/target/product/$(PRODUCT)/obj/lib/crtbegin_dynamic.o \
         $(Android_PATH)/prebuilt/darwin-x86/toolchain/arm-eabi-4.4.3/lib/gcc/arm-eabi/4.4.3/libgcc.a\
         -lc -lm
                  
IGNORE = -D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__\
         -DANDROID -DSK_RELEASE -DNDEBUG  -UDEBUG -march=armv5te -mtune=xscale\
         -msoft-float -mfloat-abi=hard


all:
    $(CPP) $(LDFLAGS) $(CFLAG) -o $(NAME) matrix_mal.cpp

要套用到自己的project只需修改

  1. Line 3: Android編譯的Product,例如如果是模擬器就把PRODUCT改成generic 
  2. Line 4: 修改Android Source Code的位置 Android_PATH
  3. Line 5: Binary的名字 NAME
  4. Line 40: Source Code的名字,替換matrix_mal.cpp
  5. Line 40如果是要編譯C code的話,將 CPP改成CC
另外,如果在編譯的時候想加入額外的參數,可以:
$ make CFLAG+="額外的參數"
或是如果想要指定binary的名稱,可以:
$ make NAME="名稱"

Cross Compile C/C++ using agcc(arm-eabi-gcc/g++ provide by Android)

如果你有Android的 Source Code 然後想要將C/C++ code  cross compile到ARM上面的話,可以使用 Android所提供的cross compiler。但有一個問題是,要真正拿來編譯的時候需要加入很多CFLAG(Include Header Path及Linking Library)。

比較方便的作法是直接使用agcc。這是外國人寫得一個perl的script,避免以後不見先貼上來。

#!/usr/bin/perl -w
use strict;

# Copyright 2008, Andrew Ross andy@plausible.org
# Distributable under the terms of the GNU GPL, see COPYING for details

# The Android toolchain is ... rough.  Rather than try to manage the
# complexity directly, this script wraps the tools into an "agcc" that
# works a lot like a gcc command line does for a native platform or a
# properly integrated cross-compiler.  It accepts arbitrary arguments,
# but interprets the following specially:
#
# -E/-S/-c/-shared - Enable needed arguments (linker flags, include
#                    directories, runtime startup objects...) for the
#                    specified compilation mode when building under
#                    android.
#
# -O - Turn on the optimizer flags used by the Dalvik build.  No
#           control is provided over low-level optimizer flags.
#
# -W - Turn on the warning flags used by the Dalvik build.  No
#           control is provided over specific gcc warning flags.
#
# Notes:
# + The prebuilt arm-eabi-gcc from a built (!) android source
#   directory must be on your PATH.
# + All files are compiled with -fPIC to an ARMv5TE target.  No
#   support is provided for thumb.
# + No need to pass a "-Wl,-soname" argument when linking with
#   -shared, it uses the file name always (so don't pass a directory in
#   the output path for a shared library!)

# Dance around to find the actual android toolchain path (it's very
# deep, so links on $PATH are going to be common.
my $GCC = `which arm-eabi-gcc`;
$GCC = qx(cd `dirname $GCC`; /bin/pwd);
chomp $GCC;
die "bad arm-eabi-gcc path" if $GCC !~ /(.*)\/prebuilt\//;
my $DROID = $1;

my $ALIB = "$DROID/out/target/product/generic/obj/lib";
my $TOOLCHAIN = "$DROID/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1";

my @include_paths = (
    "-I$DROID/system/core/include",
    "-I$DROID/hardware/libhardware/include",
    "-I$DROID/hardware/ril/include",
    "-I$DROID/dalvik/libnativehelper/include",
    "-I$DROID/frameworks/base/include",
    "-I$DROID/external/skia/include",
    "-I$DROID/out/target/product/generic/obj/include",
    "-I$DROID/bionic/libc/arch-arm/include",
    "-I$DROID/bionic/libc/include",
    "-I$DROID/bionic/libstdc++/include",
    "-I$DROID/bionic/libc/kernel/common",
    "-I$DROID/bionic/libc/kernel/arch-arm",
    "-I$DROID/bionic/libm/include",
    "-I$DROID/bionic/libm/include/arch/arm",
    "-I$DROID/bionic/libthread_db/include",
    "-I$DROID/bionic/libm/arm",
    "-I$DROID/bionic/libm",
    "-I$DROID/out/target/product/generic/obj/SHARED_LIBRARIES/libm_intermediates");

my @preprocess_args = (
    "-D__ARM_ARCH_5__",
    "-D__ARM_ARCH_5T__",
    "-D__ARM_ARCH_5E__",
    "-D__ARM_ARCH_5TE__", # Already defined by toolchain
    "-DANDROID",
    "-DSK_RELEASE",
    "-DNDEBUG",
    "-include", "$DROID/system/core/include/arch/linux-arm/AndroidConfig.h",
    "-UDEBUG");

my @warn_args = (
    "-Wall",
    "-Wno-unused", # why?
    "-Wno-multichar", # why?
    "-Wstrict-aliasing=2"); # Implicit in -Wall per texinfo

my @compile_args = (
    "-march=armv5te",
    "-mtune=xscale",
    "-msoft-float",
    "-mthumb-interwork",
    "-fpic",
    "-fno-exceptions",
    "-ffunction-sections",
    "-funwind-tables", # static exception-like tables
    "-fstack-protector", # check guard variable before return
    "-fmessage-length=0"); # No line length limit to error messages

my @optimize_args = (
    "-O2",
    "-finline-functions",
    "-finline-limit=300",
    "-fno-inline-functions-called-once",
    "-fgcse-after-reload",
    "-frerun-cse-after-loop", # Implicit in -O2 per texinfo
    "-frename-registers",
    "-fomit-frame-pointer",
    "-fstrict-aliasing", # Implicit in -O2 per texinfo
    "-funswitch-loops");

my @link_args = (
    "-Bdynamic",
    "-Wl,-T,$DROID/build/core/armelf.x",
    "-Wl,-dynamic-linker,/system/bin/linker",
    "-Wl,--gc-sections",
    "-Wl,-z,nocopyreloc",
    "-Wl,--no-undefined",
    "-Wl,-rpath-link=$ALIB",
    "-L$ALIB",
    "-nostdlib",
    "$ALIB/crtend_android.o",
    "$ALIB/crtbegin_dynamic.o",
    "$TOOLCHAIN/lib/gcc/arm-eabi/4.2.1/interwork/libgcc.a",
    "-lc",
    "-lm");
    
# Also need: -Wl,-soname,libXXXX.so
my @shared_args = (
    "-nostdlib",
    "-Wl,-T,$DROID/build/core/armelf.xsc",
    "-Wl,--gc-sections",
    "-Wl,-shared,-Bsymbolic",
    "-L$ALIB",
    "-Wl,--no-whole-archive",
    "-lc",
    "-lm",
    "-Wl,--no-undefined",
    "$TOOLCHAIN/lib/gcc/arm-eabi/4.2.1/interwork/libgcc.a",
    "-Wl,--whole-archive"); # .a, .o input files go *after* here

# Now implement a quick parser for a gcc-like command line

my %MODES = ("-E"=>1, "-c"=>1, "-S"=>1, "-shared"=>1);

my $mode = "DEFAULT";
my $out;
my $warn = 0;
my $opt = 0;
my @args = ();
my $have_src = 0;
while(@ARGV) {
    my $a = shift;
    if(defined $MODES{$a}) {
 die "Can't specify $a and $mode" if $mode ne "DEFAULT";
 $mode = $a;
    } elsif($a eq "-o") {
 die "Missing -o argument" if !@ARGV;
 die "Duplicate -o argument" if defined $out;
 $out = shift;
    } elsif($a =~ /^-W.*/) {
 $warn = 1;
    } elsif($a =~ /^-O.*/) {
 $opt = 1;
    } else {
 if($a =~ /\.(c|cpp|cxx)$/i) { $have_src = 1; }
 push @args, $a;
    }
}

my $need_cpp = 0;
my $need_compile = 0;
my $need_link = 0;
my $need_shlink = 0;
if($mode eq "DEFAULT") { $need_cpp = $need_compile = $need_link = 1; }
if($mode eq "-E") { $need_cpp = 1; }
if($mode eq "-c") { $need_cpp = $need_compile = 1; }
if($mode eq "-S") { $need_cpp = $need_compile = 1; }
if($mode eq "-shared") { $need_shlink = 1; }

if($have_src and $mode ne "-E") { $need_cpp = $need_compile = 1; }

# Assemble the command:
my @cmd = ("arm-eabi-gcc");
if($mode ne "DEFAULT") { @cmd = (@cmd, $mode); }
if(defined $out) { @cmd = (@cmd, "-o", $out); }
if($need_cpp) { @cmd = (@cmd, @include_paths, @preprocess_args); }
if($need_compile){
    @cmd = (@cmd, @compile_args);
    if($warn) { @cmd = (@cmd, @warn_args); }
    if($opt) { @cmd = (@cmd, @optimize_args); }
}
if($need_link) { @cmd = (@cmd, @link_args); }
if($need_shlink) { @cmd = (@cmd, @shared_args); }
@cmd = (@cmd, @args);

#print join(" ", @cmd), "\n"; # Spit it out if you're curious
exec(@cmd);
但是,有一個問題是他很多的變數都是寫死得,因此有幾個地方需要修改。下面是我修改的地方。要注意的是,我下面的修改是要編譯C++的程式,如果是要編譯C的話,要記得把g++改成gcc。

  1. Line 37:將路徑修改成你電腦上的arm-eabi-g++的位置
      $GCC = "/opt/compal/android/15r1/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-g++";
  2. Line 40:將$DROID改成你電腦上Android的位置
      my $DROID = "/opt/compal/android/15r1";
  3. Line 42:一般來說,如果是build模擬器的話可以把cardhu改成generic
      my $ALIB = "$DROID/out/target/product/cardhu/obj/lib";
  4. Line 43:一樣修改arm-eabi-g++的位置
      my $TOOLCHAIN = "/opt/compal/android/15r1/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-g++";
  5. Line 119:  
      "/opt/compal/android/15r1/prebuilt/darwin-x86/toolchain/arm-eabi-4.4.3/lib/gcc/arm-eabi/4.4.3/libgcc.a",
  6. Line 134:  d
      "/opt/compal/android/15r1/prebuilt/darwin-x86/toolchain/arm-eabi-4.4.3/lib/gcc/arm-eabi/4.4.3/libgcc.a",

2012年5月30日 星期三

EDK2(Tianocore) with VC2012

EDK2是Intel 的UEFI實作 open source出來的成果。目前的build script裡面還沒把VC2012加進去。 以下簡單說明抓取EDK2到怎麼加入一個toolchain然後compile。

  • 下載EDK2
      我是直接透過svn來fetch code。 svn co https://edk2.sourceforge.net/svnroot/edk2 edk2
  • 修改Build Script
      抓下來後我們修改的地方是trunk\edk2
      1. edksetup.bat
          這邊要把VS110COMNTOOLS這個環境變數加上去,圖中圈起來的部分就是額外加入的部分。
          if defined VS110COMNTOOLS (
              call "%VS110COMNTOOLS%\vsvars32.bat"
            ) else (
          
      2. Conf/tools_def.txt
          加入的地方分成三個部分。
          1. 在 Line49的地方加入
              DEFINE VS2012x86_BIN    = C:\Program Files (x86)\Microsoft Visual Studio 11.0\Vc\bin
              DEFINE VS2012x86_DLL    = C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE;DEF(VS2012x86_BIN)
              DEFINE VS2012x86_BINX64 = DEF(VS2012x86_BIN)\x86_amd64
              DEFINE VS2012x86_BIN64  = DEF(VS2012x86_BIN)\x86_ia64
              
          2. 在Line 479的地方加入
              #  VS2005x86xASL -win64- Requires:
              #                             Microsoft Visual Studio 2005 Team Suite Edition (x86)
              #                             Microsoft Windows Server 2003 Driver Development Kit(Microsoft WINDDK) version 3790.1830 
              #                        Optional:
              #                             Required to build EBC drivers:
              #                               Intel(r) Compiler for Efi Byte Code (Intel(r) EBC Compiler)
              #                             Required to build platforms or ACPI tables:
              #                               Microsoft ASL ACPI Compiler (asl.exe) v4.0.0 from
              #                               http://download.microsoft.com/download/2/c/1/2c16c7e0-96c1-40f5-81fc-3e4bf7b65496/microsoft_asl_compiler-v4-0-0.msi
              
          3. 第三個比較長,我是貼在2299行,也就是上一段IPF Definition結束的地方。
              ####################################################################################
              #   VS2012x86xASL     - Microsoft Visual Studio 2012 with Microsoft ASL
              #   ASL  - Microsoft ACPI Source Language Compiler (asl.exe)
              ####################################################################################
              *_VS2012x86xASL_*_*_FAMILY        = MSFT
              
              *_VS2012x86xASL_*_MAKE_PATH       = DEF(VS2012x86_BIN)\nmake.exe
              *_VS2012x86xASL_*_MAKE_FLAG       = /nologo
              *_VS2012x86xASL_*_RC_PATH         = DEF(WINSDK_BIN)\rc.exe
              
              *_VS2012x86xASL_*_MAKE_FLAGS      = /nologo
              *_VS2012x86xASL_*_SLINK_FLAGS     = /NOLOGO /LTCG
              *_VS2012x86xASL_*_APP_FLAGS       = /nologo /E /TC
              *_VS2012x86xASL_*_PP_FLAGS        = /nologo /E /TC /FIAutoGen.h
              *_VS2012x86xASL_*_VFRPP_FLAGS     = /nologo /E /TC /DVFRCOMPILE /FI$(MODULE_NAME)StrDefs.h
              
              *_VS2012x86xASL_*_ASM16_PATH      = DEF(VS2012x86_BIN)\ml.exe
              
              ##################
              # ASL definitions
              ##################
              *_VS2012x86xASL_*_ASL_PATH        = DEF(WIN_ASL_BIN)
              *_VS2012x86xASL_*_ASL_FLAGS       = DEF(MS_ASL_FLAGS)
              *_VS2012x86xASL_*_ASL_OUTFLAGS    = DEF(MS_ASL_OUTFLAGS)
              *_VS2012x86xASL_*_ASLCC_FLAGS     = DEF(MSFT_ASLCC_FLAGS)
              *_VS2012x86xASL_*_ASLPP_FLAGS     = DEF(MSFT_ASLPP_FLAGS)
              *_VS2012x86xASL_*_ASLDLINK_FLAGS  = DEF(MSFT_ASLDLINK_FLAGS)
              
              ##################
              # IA32 definitions
              ##################
              *_VS2012x86xASL_IA32_*_DLL        = DEF(VS2012x86_DLL)
              
              *_VS2012x86xASL_IA32_MAKE_PATH    = DEF(VS2012x86_BIN)\nmake.exe
              *_VS2012x86xASL_IA32_CC_PATH      = DEF(VS2012x86_BIN)\cl.exe
              *_VS2012x86xASL_IA32_VFRPP_PATH   = DEF(VS2012x86_BIN)\cl.exe
              *_VS2012x86xASL_IA32_ASLCC_PATH   = DEF(VS2012x86_BIN)\cl.exe
              *_VS2012x86xASL_IA32_ASLPP_PATH   = DEF(VS2012x86_BIN)\cl.exe
              *_VS2012x86xASL_IA32_SLINK_PATH   = DEF(VS2012x86_BIN)\lib.exe
              *_VS2012x86xASL_IA32_DLINK_PATH   = DEF(VS2012x86_BIN)\link.exe
              *_VS2012x86xASL_IA32_ASLDLINK_PATH= DEF(VS2012x86_BIN)\link.exe
              *_VS2012x86xASL_IA32_APP_PATH     = DEF(VS2012x86_BIN)\cl.exe
              *_VS2012x86xASL_IA32_PP_PATH      = DEF(VS2012x86_BIN)\cl.exe
              *_VS2012x86xASL_IA32_ASM_PATH     = DEF(VS2012x86_BIN)\ml.exe
              
                    *_VS2012x86xASL_IA32_MAKE_FLAGS  = /nologo
                DEBUG_VS2012x86xASL_IA32_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1ib2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm
              RELEASE_VS2012x86xASL_IA32_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1ib2 /GL /FIAutoGen.h /EHs-c- /GR- /GF
              NOOPT_VS2012x86xASL_IA32_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Od
              
                DEBUG_VS2012x86xASL_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
              RELEASE_VS2012x86xASL_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd
              NOOPT_VS2012x86xASL_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
              
                DEBUG_VS2012x86xASL_IA32_DLINK_FLAGS = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /MAP /ALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /MACHINE:X86 /LTCG /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DRIVER /DEBUG
              RELEASE_VS2012x86xASL_IA32_DLINK_FLAGS = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /IGNORE:4254 /OPT:REF /OPT:ICF=10 /MAP /ALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /MACHINE:X86 /LTCG /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DRIVER /MERGE:.data=.text /MERGE:.rdata=.text
              NOOPT_VS2012x86xASL_IA32_DLINK_FLAGS   = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /MAP /ALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /MACHINE:X86 /LTCG /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DRIVER /DEBUG
              
              ##################
              # X64 definitions
              ##################
              *_VS2012x86xASL_X64_*_DLL         = DEF(VS2012x86_DLL)
              
              *_VS2012x86xASL_X64_CC_PATH       = DEF(VS2012x86_BINX64)\cl.exe
              *_VS2012x86xASL_X64_PP_PATH       = DEF(VS2012x86_BINX64)\cl.exe
              *_VS2012x86xASL_X64_APP_PATH      = DEF(VS2012x86_BINX64)\cl.exe
              *_VS2012x86xASL_X64_VFRPP_PATH    = DEF(VS2012x86_BINX64)\cl.exe
              *_VS2012x86xASL_X64_ASLCC_PATH    = DEF(VS2012x86_BINX64)\cl.exe
              *_VS2012x86xASL_X64_ASLPP_PATH    = DEF(VS2012x86_BINX64)\cl.exe
              *_VS2012x86xASL_X64_ASM_PATH      = DEF(VS2012x86_BINX64)\ml64.exe
              *_VS2012x86xASL_X64_SLINK_PATH    = DEF(VS2012x86_BINX64)\lib.exe
              *_VS2012x86xASL_X64_DLINK_PATH    = DEF(VS2012x86_BINX64)\link.exe
              *_VS2012x86xASL_X64_ASLDLINK_PATH = DEF(VS2012x86_BINX64)\link.exe
              
                DEBUG_VS2012x86xASL_X64_CC_FLAGS     = /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /O1ib2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm
              RELEASE_VS2012x86xASL_X64_CC_FLAGS     = /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /O1ib2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF
              NOOPT_VS2012x86xASL_X64_CC_FLAGS       = /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm /Od
              
                DEBUG_VS2012x86xASL_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
              RELEASE_VS2012x86xASL_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
              NOOPT_VS2012x86xASL_X64_ASM_FLAGS      = /nologo /c /WX /W3 /Cx /Zd /Zi
              
                DEBUG_VS2012x86xASL_X64_DLINK_FLAGS  = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /MAP /ALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /Machine:X64 /LTCG /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DRIVER /DEBUG
              RELEASE_VS2012x86xASL_X64_DLINK_FLAGS  = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /IGNORE:4254 /OPT:REF /OPT:ICF=10 /MAP /ALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /Machine:X64 /LTCG /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DRIVER /MERGE:.data=.text /MERGE:.rdata=.text
              NOOPT_VS2012x86xASL_X64_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT:ICF=10 /MAP /ALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /Machine:X64 /LTCG /DLL /ENTRY:$(IMAGE_ENTRY_POINT) /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DRIVER /DEBUG
              
      3. 編譯
          1. edksetup.bat --nt32
          2. build -t VS2012x86xASL
      4. 啟動模擬器
          build -t VS2012x86xASL run
  • 2012年5月25日 星期五

    UEFI/PI (7) UEFI Firmware Hierarchy


  • UEFI Firmware Hierarchy
      1. Firmware Storage
          Firmware Storage 是非揮發性的memory儲存BISO或Firmware的code。UEFI的其中一個優勢是DXE和PEI 可以把code解壓到firmware storage外。 Firmware Storage最小的單位是UEFI Firmware File,再來是Firmware File System(FFS)、Firmware Volumn(FV)及Firmware Device(FD)。
          • UEFI Firmware Files
              UEFI Firmware Files是最小/少的模組化的code,包含了Code及Data存放在FV中。每一個UEFI Firmware File都有Name、Type、Alignment及Size等四個attribute。 UEFI build tool會建立Firmware Files。Firmware Files的組成如下圖:
                其中,entry point包含在Data區塊裡頭。
            • Firmware File System
                Firmware File System(FFS)描述FV的file及free space的資訊。每一個FFS都有一個unique的GUID來連結FV。FFS參考FAT 32的作法,他的header如下圖。
                  同時,我們也利用UEFI Build tool建立FFS。
              • Firmware Volumes
                  一個Firmware Volumes對應著一個logical firmware device。FV是由FFS來組成的,Firmware file則是FV組成的元件。FV可以表示flash的一部分或是多個flash或甚至一個partition或是network。所有PI component都存放在FV,在PI的過程中,必須透過FFS去讀取。
                • Firmware Devices
                    Firmware Device是指非揮發性的Storage。包含了Firmware Code、Firmware Data及Fault Tolerant。
                      Firmware Device包含了FV Main放置了DXE Driver,FV Recovery放置了security及PEI code。我們以Recovery FV為例作解釋。Recovery FV包含了FFS放置著Boot Block Code(包含了PEI security phase code) 及Recovery code(包含了也許USB stack來讀取USB device)。
                  • Creating UEFI Firmware Files
                      我們如何產生UEFI Image呢?
                      1. 撰寫C code
                      2. 用build tool編譯產生Library及object file
                      3. 產生的Library及object file是intermediate file,利用微軟的compiler把他們轉成PE/COFF格式的image。
                      4. 如果是在Linux的環境,我們可以使用GCC來取代。
                      5. Build Tool會將標準的header置換成UEFI header。
                      在PEI階段,因為我們是run在rom裡面,因此我們希望Image越小越好。所以我們是製作成Terse Image。它簡化了UEFI header的項目如下圖。

                  • Physical Memory Map
                      從下圖我們可以看到各個層級之間的關係。Firmware Device被load進system memory之後,它包含了firmware volume。而firmware volume又包含了許多Firmware File System,Firmware File System又放置著許多Firmware File。

              • UEFI/PI (6) UEFI Drivers



              • UEFI Drivers and Overview
                  1. Driver's Location
                      UEFI Driver run在DXE Phase中,在platform 初始化的過程中可能重複的被active。
                       
                      同時,UEFI Driver是green H的其中一部份,如下圖所示。 


                  2. UEFI Driver's Attributes
                      UEFI Driver是system driven的image。在DXE Phase中由UEFI Loader到系統中。UEFI Driver能提供protocols同時也可能使用其他Driver提供的protocol。我們可以使用UEFI Driver去支援特定硬體或override 已經存在的Driver。
                      1. Supportive
                          UEFI Driver支援複雜的bus hierachies。UEFI Driver可以與bus interface溝通來將device連接到bus。
                      2. Independent
                          UEFI Driver可以存放在任意的storage 中 包含flash。
                      3. Flexible
                          其中一個 UEFI Driver會建立UEFI Driver Binding Protocol.這一個protocol是UEFI定義的protocol用來將Driver與device做配對。同時也支援version info、hot-plug及unload的功能。
                        • Extensible
                            UEFI Driver被設計成可以支援未知的bus或device的種類。
                    • UEFI Driver's Functions

                        1. UEFI Driver是firmware的延伸
                            可以支援新的硬體,與HW、OS無關。
                        2. Portability
                            因為與Platform Arch.無關,因此可以跨平台。
                        3. UEFI SPEC.提供許多API
                            因此可以加速開發。
                    • UEFI Driver's Contents
                        UEFI Driver包含了四大部分,
                        1.會有一個Entry Point作為Driver的進入點。
                        2.會提供Published Function 供其他的Driver使用。
                        3.Consumed Function,參考別的Driver提供出來的功能。
                        4.Data Structure,用來滿足Driver所需要的資料存放。 

                    • UEFI Driver's Entry Point
                        如果前面所提到的,DXE Dispatcher會Load Driver,UEFI Driver透過UEFI Loader將Driver Load到記憶體中。Loader 會先進入Driver的Entry Point,接著Driver會建立Published Protocol。以下圖為例,他提供了Binding Protocol( supported, Start, Stop),最後離開。

                    • Drivers v.s. Applications
                        UEFI Application 與 UEFI Driver本質上非常的類似,下表列出他們的比較。 



                • UEFI Protocols
                    1. UEFI Protocols
                        一個UEFI protocol定義在SPEC中,可能是一塊function pointer、data structure或是API。 下面分成三個部份來解釋。
                        1. Confused with Drivers
                            UEFI Driver是一個可執行的UEFI Image。它包含了許多的protocol而每一個protcol也包含了多個handle。因此,Protocl是Driver建立的。
                            我們可以把UEFI protocl想像成interface。它包含了許多function pointer, data structure或APIs。
                        2. Produced by Driver
                            UEFI Driver可以建立許多protocol。
                        3. Consumed by Anyone
                            UEFI protocol可能被任一的Driver使用。例如UEFI platform Driver可能在其他的Driver啟動時使用。
                    2. Example A: EFI_PCI_IO_PROTOCOL
                        下面的例子是PCI I/O Protocol。這一個Driver run在UEFI boot services/environment來存取memory及PCI controler的I/O。另外,與PCI相關的SPEC也定義在此protocol中。這一個Protocol的目的是要將對PCI device的讀寫抽象化。EFI_PCI_IO_PROTOCOL抽象化了對PCI 裝置的memory,I/O, PCI configuration及DMA interface的操作。每一個PCI Bus的PCI control都有對應的一個EFI_PCI_IO_PROTOCOL instance。對操作PCI裝置的Driver都必須透過對應的instance來操作。PCI controler的handle至少會有EFI_PATH_PROTOCOL及EFI_PCI_IO_PROTOCOL的instance。

                    3. Example B
                        EFI_DISK_IO_PROTOCOL是對block device存取抽象化。File System等等的code都需要依賴這個protocol。

                    4. Example C
                        這個protocl的解釋如下:UEFI Image may use the device path to match its own device Drivers to the particular device. Note that the executing UEFI OS loader and UEFI application images must access all physical devices via Boot Services device handles until ExitBootServices() is successfully called.



                • Driver Design
                    1. Design Process Steps

                    2. Driver Types
                        在撰寫Driver之前,必須先知道我們要寫的是哪一種Driver。在UEFI Image中可以分成兩種,Driver及Application。
                        UEFI Applications中有一種特殊的application叫做OS Loader。與一般的Application不同的是OS Loader會呼叫 ExitBootService然後就將control pass到OS kernel了。
                        接著我們來了解Driver中的幾種類型。
                        1. Service Driver
                            a. 不管理hardware。 
                            b.提供service給其他Driver。 
                            c.不支援Driver binding protocol。 
                            d. 在entry point裡install protocol。  
                            e. 建立一個或多個 service handlers。 
                            f. 產生service-specific protocols。 
                            例如: UEFI Decompress Protocol、UEFI Byte Code Virtual Machine、Boot Integrity Service(BIS)。
                        2. Initialization Drivers
                            a. 與hardware會有溝通。 
                            b. one-time initialization。 
                            c. 不會建立handle及protocol。 
                            d. 結束後就會unload。
                        3. Root Bridge Driver
                            a. 管理部分的core chipset。 
                            b. 直接與HW溝通。 
                            c. 建立一個或多個 root bridge handle。 
                            d. 建立 root bridge I/O protocols,安裝在新的root bridge handle。 例如:PCI Host Bridge。
                        4. Bus Driver
                            他是UEFI Driver Model的Driver。用來管理Bus controller。
                            start()會建立一個或多個child handles及Bus Specific I/O protocols。例如:PCI NIC card、UART controllers。
                        5. Device Driver
                            管理controller或peripheral device。
                            start()不會建立child handle但會建立一個或多個protocols。
                            例如:PCI Video Adapter、USB Host Controller、USB keyboard/Mice。
                        6. Hybrid Driver
                            結合Bus Driver及device Driver。
                            例如:PCI SCSI Host Controller、PCI Fiber Channel Controller。
                    3. Consumed Protocols
                        什麼時候會使用那些Protocl呢?例如 PCI Adapter需要PCI I/O Protocol或加上也許Device Path protocol。 而像USB keyboard/mice, DVD rom這類的裝置就需要USB I/O protocol及device path protocol。
                    4. Produces Protocols
                        那我們可能會產生什麼protocol呢?例如,假設是在寫keyboard Driver,我們可能會建立simple input protocol;滑鼠可能會有simple pointer protocol;USB flash disk可能會有block I/O protocol。Depends on 你寫的裝置。



                • Writing Drivers
                    1. Writing UEFI Drivers
                        接下來,我們寫一個Driver需要那些protocol呢?參考下表:

                    2. Initialized
                        當UEFI Driver被dispatch時,首先會進入entry point然後初始化。在初始化的過程中會宣告此Driver會提供那些protocol。UEFI Driver 是relocatable pickup image,所以可以load在記憶體的任意位置。在Driver initialize的過程中是不會touch HW的,在過程中會呼叫Driver binding protocol,在初始化後,service就被註冊了,這也是會什麼可以快速開機的原因。
                    3. Supported
                        在初始化後,Binding Protocol會開始define Supported、Start及Stop function。
                        接下來以PCI Driver為例。 首先Open PCI Protocol( EFI_PCI_IO_PROTOCOL),然後check這個Driver是否支援這個controller( EFI_DEVICE_PATH_PROTOCOL)再來關閉PCI Protcol並return 是否支援。


                    4. Start
                        Start()會產生EFI_Block_IO_Protocol 建立child protocol(functions)。

                    5. Stop
                        Stop不會產生任何的Protocol,反而他會移除Start()所產生的Protocol。

                    6. Recommended Protocols


                • Tips and Techniques

                    1. Driver Guideline
                        主要有四個原則
                        1. 在Driver entry的地方不要touch HW
                        2. 在Support()保持簡單。
                        3. Start()對應Stop();Driver Entry對應Unload
                        4. 把複雜的事情放到Start()和Stop()去做。
                    2. Design Checklist

                    3. Recommendations
                        在實作Test/Debug的時候有幾點注意的事情如下:

                    4. Start() Code
                        這個範例是儲存PCI的Attributes。

                    5. Stop() Code
                        將Attribute reset回原本的值。

                    6. Library Functions
                        我們應該多利用既有的function來將code size降低,如下:

                    7. Other Helpful Suggestions
                        這裡分成兩大塊。
                        Reducing Option ROM Size
                        1. 使用UEFI支援的Compression來降低ROM Size。
                        2. Compile with EFI Byte Code Compiler(EBC)

                        Improving Protability
                        1. 不要假設max number of children。
                        2. 不要寫死memory address或使用assembly。
                        3. 不要使用floating point arithmetic
                        4. Dome minor ebc point considerations
                        5. Bus Driver應該支援一次產生一個child。(可以加速開機)