mirror_edk2/BaseTools/Source/Python/GenFds
Laszlo Ersek bd907fb638 BaseTools/GenFds: speed up Region.PadBuffer()
The current implementation calls both pack() and Buffer.write() Size
times. The new implementation calls both of these methods only once; the
full data to write are constructed locally [1]. The range() function is
replaced by xrange() because the latter is supposed to be faster / lighter
weight [2].

On my laptop, I tested the change as follows: I pre-built the series at
[3] with

  build -a X64 -p OvmfPkg/OvmfPkgX64.dsc -t GCC48 -b DEBUG \
      -D HTTP_BOOT_ENABLE -D SECURE_BOOT_ENABLE

(The series at [3] is relevant because it increases the size of one of the
padded regions by 8.5 MB, slowing down the build quite a bit.)

With all source code already compiled, repeating the above command takes
approximately 45 seconds. With the patch applied, it goes down to 29
seconds.

[1] http://stackoverflow.com/questions/27384093/fastest-way-to-write-huge-data-in-file
[2] https://docs.python.org/2/library/functions.html?highlight=xrange#xrange
[3] http://thread.gmane.org/gmane.comp.bios.edk2.devel/14214

We can also measure the impact with a synthetic test:

> import timeit
>
> test_old = """
> import struct, string, StringIO
> Size = (8 * 1024 + 512) * 1024
> Buffer = StringIO.StringIO()
> PadData = 0xFF
> for i in range(0, Size):
>     Buffer.write(struct.pack('B', PadData))
> """
>
> test_new = """
> import struct, string, StringIO
> Size = (8 * 1024 + 512) * 1024
> Buffer = StringIO.StringIO()
> PadByte = struct.pack('B', 0xFF)
> PadData = string.join(PadByte for i in xrange(0, Size))
> Buffer.write(PadData)
> """
>
> print(timeit.repeat(stmt=test_old, number=1, repeat=3))
> print(timeit.repeat(stmt=test_new, number=1, repeat=3))

The output is

[8.231637001037598, 8.81188416481018, 8.948754072189331]
[0.5503702163696289, 0.5461571216583252, 0.578315019607544]

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2016-07-12 13:19:27 +02:00
..
__init__.py Sync EDKII BaseTools to BaseTools project r1971 2010-05-18 05:04:32 +00:00
AprioriSection.py There is a limitation on WINDOWS OS for the length of entire file path can’t be larger than 255. There is an OS API provided by Microsoft to add “\\?\” before the path header to support the long file path. Enable this feature on basetools. 2014-08-15 03:06:48 +00:00
Attribute.py Sync EDKII BaseTools to BaseTools project r1971 2010-05-18 05:04:32 +00:00
Capsule.py BaseTools: Supported FMP capsule image. 2015-06-23 06:46:01 +00:00
CapsuleData.py BaseTools: Supported FMP capsule image. 2015-06-23 06:46:01 +00:00
ComponentStatement.py Sync EDKII BaseTools to BaseTools project r1971 2010-05-18 05:04:32 +00:00
CompressSection.py License header updated to match correct format. 2014-08-28 13:53:34 +00:00
DataSection.py There is a limitation on WINDOWS OS for the length of entire file path can’t be larger than 255. There is an OS API provided by Microsoft to add “\\?\” before the path header to support the long file path. Enable this feature on basetools. 2014-08-15 03:06:48 +00:00
DepexSection.py License header updated to match correct format. 2014-08-28 13:53:34 +00:00
EfiSection.py BaseTools: Fixed incorrect alignment bug. 2015-07-16 00:56:14 +00:00
Fd.py There is a limitation on WINDOWS OS for the length of entire file path can’t be larger than 255. There is an OS API provided by Microsoft to add “\\?\” before the path header to support the long file path. Enable this feature on basetools. 2014-08-15 03:06:48 +00:00
FdfParser.py BaseTools: fix the bug for FMP to support use Macro as path description 2016-04-29 14:51:06 +08:00
Ffs.py Sync BaseTools Trunk (version r2518) to EDKII main trunk. 2012-04-10 07:18:20 +00:00
FfsFileStatement.py BaseTools/GenFds: Fix the bug for wrong alignment generate for RAW file 2016-04-05 13:24:04 +08:00
FfsInfStatement.py BaseTools/GenFds: enhance INF built arch filter 2016-05-16 09:03:41 +08:00
Fv.py BaseTools/GenFds: Fix the bug for wrong alignment generate for RAW file 2016-04-05 13:24:04 +08:00
FvImageSection.py BaseTools: Fix bug in GenFds to handle FV image alignment 2016-05-11 12:25:44 +08:00
GenFds.py BaseTools/GenFds: enhance to get TOOL_CHAIN_TAG and TARGET value 2016-05-25 13:43:54 +08:00
GenFdsGlobalVariable.py BaseTools: add the support for --pcd feature to patch the binary efi 2016-04-20 09:32:52 +08:00
GuidSection.py BaseTools: cache the defined Guid tool to improve the performance 2016-04-06 13:40:14 +08:00
OptionRom.py There is a limitation on WINDOWS OS for the length of entire file path can’t be larger than 255. There is an OS API provided by Microsoft to add “\\?\” before the path header to support the long file path. Enable this feature on basetools. 2014-08-15 03:06:48 +00:00
OptRomFileStatement.py There is a limitation on WINDOWS OS for the length of entire file path can’t be larger than 255. There is an OS API provided by Microsoft to add “\\?\” before the path header to support the long file path. Enable this feature on basetools. 2014-08-15 03:06:48 +00:00
OptRomInfStatement.py Sync BaseTools Branch (version r2362) to EDKII main trunk. 2011-10-11 02:49:48 +00:00
Region.py BaseTools/GenFds: speed up Region.PadBuffer() 2016-07-12 13:19:27 +02:00
Rule.py Sync EDKII BaseTools to BaseTools project r1971 2010-05-18 05:04:32 +00:00
RuleComplexFile.py Sync EDKII BaseTools to BaseTools project r1971 2010-05-18 05:04:32 +00:00
RuleSimpleFile.py Sync EDKII BaseTools to BaseTools project r1971 2010-05-18 05:04:32 +00:00
Section.py BaseTools: Fix GenFds issue to wrongly get file without postfix. 2016-05-26 10:58:30 +08:00
UiSection.py BaseTools: Clean some coding style issues 2015-12-01 04:22:16 +00:00
VerSection.py BaseTools: Clean some coding style issues 2015-12-01 04:22:16 +00:00
Vtf.py BaseTools: Clean some coding style issues 2015-12-01 04:22:16 +00:00