Tuesday, December 24, 2013

Technical Communications

My last post talked about invention and innovation http://vzimmer.blogspot.com/2013/12/invention-and-innovation.html, and in that post I mentioned an article from the Harvard Business Review. Touching on such business aspects in a blog with the theme of 'musing on technology' may appear to be a category error, but for me, business underlies many of our technical endeavors.

Specially, I believe that technology is really about people, and people interact via the macro-economy via business, so they are all related in my eyes. A strict logician could argue via parody of my assertion with the following: 'technology is made of atoms, and atoms interact via the laws of quantum mechanics, so where are your QM postings?' My reply to such a syllogism would be 'the post is upcoming.'

So enough preamble, let me explore what I mean by technical communications in this post. A few recent events inspired this posting. The first was a discussion at Seatac airport http://www.portseattle.org/sea-tac/Pages/default.aspx with a former Intel manager. We were both waiting to fly to San Francisco, and our flight was delayed by a couple of hours. Fog in San Francisco, who would have thought it possible? This manager now works for the hardware division of Amazon, namely Lab 126 http://www.lab126.com, or "A to Z." I asked him about the admonition by Amazon's CEO Jeff Bezos against misusing Power Point that I had read about on the web http://conorneill.com/2012/11/30/amazon-staff-meetings-no-powerpoint. He told me that the practice of writing white papers ahead of the meeting and using Power Point as 'speaker notes' finds practice throughout all of Amazon's groups. The manta of "think complex, speak simple" summarizes the intent of the behavior.

The ex-Intel manager brought the Amazon culture close to home for me when he said "Don't you realize how many presentations get stuck on discussing a bullet for the whole meeting? At Amazon, we can deflect such delays by referencing the white paper, for example." In my professional career where white papers are not fully embraced, I try to avoid the single bullet rat hole with 'lap rules.' The term 'rate hole' and 'rat holing' is common in the tech industry and described well in Johnson's book Absolute Honesty http://www.amazon.com/Absolute-Honesty-Building-Corporate-Integrity/dp/0814407811. To avoid a rate hole via lap rules I advocate the following, 'Lap #1' allows the speaker to present all of his material without interruption. 'Lap #2' is a re-review of the same slide deck, which allows for questions. Regrettably, many senior people cannot restrain themselves and will camp on a bullet, or even the title, during 'Lap #1.'

When it comes to rat holes, the "Highest Paid Person's Opinion" (HIPPO) http://www.forbes.com/sites/derosetichy/2013/04/15/what-happens-when-a-hippo-runs-your-company/ is especially prone to this behavior of jumping to early conclusions prior to hearing a full review. I once asked a HIPPO after a meeting if he/she really thought that being the most senior person justified the enforcement of an opinion without having all of the data, and the reply was "of course or else the company wouldn't pay me this much." Interesting observation and maybe sour grapes on my sub-HIPPO status, but I encourage such parties to temper that alacrity with the possibility of succumbing to the logical fallacy of confusing correlation ("I get paid a lot so I am right") and causality ("I reviewed the data and with my experience I assert that I am right"). To combat the attraction of correlation-based reasoning I advocate a bit more Socratic questioning in the venues where seniority provides access.

From my last posting I talked about the 'exit champion,' or the pejorative characterization of such parties as "Dr. No" or the "No-Bots", so a cocktail of a 'HIPPO plus No-Bot' may engage the most vigorously in the rat-holing. Or even worse, the trifecta of 'HIPPO + No-Bot + Architecture Astronaut http://www.joelonsoftware.com/articles/fog0000000018.html.'

Maybe some of this crazed behavior within companies can be explained by the theory of canine-stacking  that a colleague recently described to me.  As Mike Rothman noted:

"I suppose the theory is simple:
     Top Dog
     Middle Dog
     Lower Dog
     Lowest Dog
Lowest dog works like crazy, but Lower dog adds a little something and communicates up what he did (inclusive of lowest dog's work) - and each layer above adds a touch of something and fronts for all the work below....You just hope that each layer had added something useful other than passing the word around...."

On a personal note, I appreciate the Amazon-eque white paper sentiment and try to use the written word as a way to scale and convey complex thoughts, strategies, and technical designs. And on the topic of scaling my impact, I hearken back to a quote from a manager a decade ago who told me "you should produce the output of as many people as your grade level." Taken literally this can be the IT equivalent of John Henry http://en.wikipedia.org/wiki/John_Henry_(folklore), I fear. It’s tough to write specifications + code at the same level of ten people, but working through others, collaborations, training, writing things down, etc, are some of the tools by which I can scale my efforts.

The next event that started me down the path of thinking about communications was a presentation Yuriy Bulygin, the Chief Threat Researcher, myself, and John Loucaides gave at Cisco SecCon http://www.cisco.com/web/about/security/cspo/csdl/seccon-overview.html. The narrative proceeded from attacks (or offense), then into technology countermeasures (or defense), and finally answered the question of what to do in case of a vulnerability (or response).

One comment from an attendee included "Quite impressive. You told an epic tale in less than an hour." To me this was a reminder that the 10's of slides and the constrained time frame for explanation plus demonstration exceeded the medium of Power Point. This class of erudition needs a complementary discourse mechanism, such as the written word.

Another recent event, though, that reminds me that I may not be following my own advice was my presentation on "Platform Firmware Security" at Seattle BSides http://www.securitybsides.com/w/page/57847942/BsidesSeattle in Redmond, WA a couple Saturdays past. Specially, I cribbed a long Power Point deck https://docs.google.com/file/d/0BxgB4JDywk3MSncxUHlIN0tYdms/edit but I didn't produce an updated white paper for offline reading. I was told that as RSA becomes more professional, Black Hat is the new RSA, Defcon is becoming the new Black Hat, and the local BSides are becoming the new Defcon.




I posted these slides during one of the subsequent talks based upon the exhortations on Twitter
"we want the slides!" "we want the slides!" chants are heard in the twitterverse ;)

After my talk, I sat in on a talk by Jack Daniel https://twitter.com/jack_daniel on presentation techniques, including the use of more graphics than text and engaging the audience. Jack was in attendance during my talk in the morning, but luckily I preceded his presentation else I would have been especially shame-faced in having delivered my verbose deck that didn't follow his guidance.

I recently realized the impact of a white paper when I saw that my 2009 IBM and Intel white paper http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.161.7603 has been included into university curricula http://www-inst.eecs.berkeley.edu/~cs194-24/sp13/index_handouts.html. That shows scaling of the written word in one instance.

On a side note, I couldn't resist Tweeting a quote from Jack Daniel #bsidesseattle on Saturday.
"I am from Texas. It is an excuse for aberrant behavior for life."
Having grown up in Houston, TX, I have to agree w/ Jack on that point.

A final event that helped inform this posting was reading Joan Magretta's book What Management Is: How it works and why it's everyone's business http://www.amazon.com/dp/B00AQKIOFC/ref=r_ea_s_t during the round trip on Amtrack to Leavenworth last weekend. This was more of a meta-business book that provided an alternate way to think about modern management and leadership, versus today's books of recipes, sound-bites and aphorisms. I especially liked the sentiment where today's manager is possibly the last domain of the generalist since the modern worker is a super-specialist who necessarily 'knows more' than his manager. This leaves the role of the manager to inspire, lead, coordinate, and synthesis the efforts of this global workforce.

Pretty interesting read.

Well, I should summarize this post by reminding myself that the written word provides scale. Conference talks are great for networking and getting new insights, but for purposes of information dissemination they have understandable limits. In 2014 I need to convert more of my slides and talks into more deliberate writing activities. When I revisit this post in 12/14 we'll see how I have done in making progress against task on this sentiment.

With that I will close today and wish everyone a Merry Christmas.


Sunday, December 15, 2013

Invention and Innovation

I recently returned from Shanghai. I was greeted by the following when I boarded the plan from LAX to SH, though.
There was little hyperbole in that headline, I'm afraid. Since my first trip to the Shanghai to work with our team members at the Cao He Jing site in 2001, and then Shanghai Mart in the early 2000's, and finally the Zizhu campus, the pollution and traffic have increased.

But what has also increased is the team's experience and capabilities. The energy of the Intel team always recharges me, and this trip was no exception.

These trips include working sessions, open forums, talks, and other instances of collaboration. One talk I always offer to the site is patent training, for an engineer's perspective. I want to make sure that the audience doesn't believe that I can provide legal advice, but what I can do is to provide guidance on the process within the company.

Since the process by engineers capture and idea and propose it for inclusion in a potential patent filing varies per company, what I'll discuss below entails more of a higher level view of how I think about the process, and innovation in general.

For me I see a serial relationship between Invention and Innovation, as follows:

INVENTION
Company feeds $’s into Engineer + Engineer’s insight in response to a Problem Statement
Results == Patents
INNOVATION
Patents + Engineering + Marketing leads to product development
Company sells products
Results == earnings of $$$$’s (or RMB, YEN, Euro...)

-- Repeat this loop of INVENTION -> INNOVATION




So this frames the action of Invention versus the broader goals of the company, such as creating products which delight and fulfill end customer needs.  The process begins with invention, using a small 'i'. Therein an engineer is faced with a problem presently, or potentially in the future, faced by the market. The engineer devises a solution, and if it is 'novel' or distinct from other practice, it may be a candidate for formally pursuing a patent, or Invention with the capital 'I.' Of course most of our engineering problem solving is invention, but it is important to note the creation of patents because of the present nature of cross-licensing, company investment preservation in its Research and Development (R&D) spend, etc. These means the evolution of the small 'i' to the big 'I,' namely transforming invention into Invention and the associated patent applications for the latter.

With invention in hand, whether small 'i' or 'I', the long path of Innovation, or working with a team within, across, and possibly beyond the company, to deliver product. If the engineering concept was truly born of a problem statement enjoyed by the market and the team can execute on the associated reduction of the design to a shipping product, and the timing of the market is right, and....including many other factors such as the phase of the moon, resultant revenue should be borne of the activity. And as each company reinvests some revenue into R&D from whence invention is incubated, the virtuous cycle continues.

And note the emphasis on 'problem statement.' For me a good problem statement that is relevant to the business can be the source of unbounded invention, and hopefully, resultant innovation.

So invention is the idea pump that can inform product design, but Innovation is the mapping of invention into the creation of products that you ship to customers and for which you get paid.

This is how I think of invention in the context of business-driven-innovation ("BDI," oh not another acronym.....). It is perhaps a bit narrow minded and parochial, but just as I cannot see myself studying theoretical physics or maths, I feel most comfortable operating in a space where the business imperatives are manifest. Sort of like how after the Cultural Revolution in China I heard that all maths were "Maoist Mathematics", such as to support manufacturing and civil engineer, not pure maths. Maybe that explains why the Alma mater of many of my Shanghai colleagues, Shanhai Jiao Tong University, literally translates to "Traffic School" (or at least that's what they've told me)? To my mother's chagrin this is also the reason that my CV doesn't include the PhD moniker, too.

The cycle between Invention and Innovation at a large company is often tempered by the role of "Exit Champions," as well defined in the Harvard Business Review article http://hbr.org/2003/02/why-bad-projects-are-so-hard-to-kill/. It continually proves a delicate balancing act to ensure that Invention and Innovation are congruent with the business exigencies, especially given the finite resources for R&D budet allocation. But with our ever-changing market and internet-time pressure, I always feel the need to co-equally role model the 'entrance champion,' or party who delivers Invention+Innovation, as much as providing the guard rails of the 'exit champion.'

Given my love of business-driven-innovation, maybe I'll miss the next Kuhn-style paradigm shift and scientific revolution http://en.wikipedia.org/wiki/The_Structure_of_Scientific_Revolutions, but this is where I feel I fit into the flow on this dynamic.  Enough said for Sunday blogging. And thanks for reading if you made it this far.

Sunday, December 8, 2013

Better living through tools

In an earlier post http://vzimmer.blogspot.com/2013/09/end-of-summer-13.html I spoke about architecture versus implementation, and the process of successively refining an architecture to some implementation artifact. I didn't elaborate upon some of the techniques to demonstrate correspondence between the design goals and the implementation behavior. In the industry this can be naively thought of as just a simple matter of testing, but I believe it goes further than that, namely the broader concern of representing architectural intent in a product.

With respect to software development and testing, though, there are many different approaches. Recent efforts like Test Driven Development (TDD) have shown promise in practice, and I enjoyed Grenning's Test Driven Development in Embedded C on the same. Similarly, books like James Whittaker's How Google Tests Software and its associated blog http://googletesting.blogspot.com/2011/01/how-google-tests-software.html provides a very pragmatic approach to the problem, namely all developers write test and a centralized test organization is both responsible for a consultative and automation/infrastructure role. In our world of UEFI and EDK2 we have the Self-Certification Tests (SCTs) http://www.uefi.org/sites/default/files/resources/UPFS11_P3_UEFI_SCT_HP_Intel.pdf.

Now lets look at the problem from a broader level, namely expressing architectural intent. To me this is nowhere more important than in the world of building trustworthy systems, especially the "Defender's Dilemma" that Jeremiah reminds us of in slide 7 of http://www.uefi.org/sites/default/files/resources/UEFI_Summerfest_2013_-_Microsoft_Hardware_Security_Test_Interface.pdf. Namely, the attacker only has to discover one flaw, whereas the defender has to provide no gaps. And it is the 'gaps' that are important in this post since flaws can span from the architecture down into the implementation.

To that end of mapping architectural intent directly to code, I have continually been intrigued by the work of Gernot Heiser http://www.cse.unsw.edu.au/~gernot/ at NICTA on Trustworthy Systems http://www.ssrg.nicta.com.au/projects/TS/. The trophy piece of that effort is seL4, a formally validated microkernel with correspondence between C code, a machine model, Haskell, and a theorem prover. This is undoubtedly a BHAG http://en.wikipedia.org/wiki/Big_Hairy_Audacious_Goal to scale more generally, but it does serve as a beacon to show what can be done given sufficient focus and incentives.

Gernot's effort is not alone, of course. There is the verification of the CMU SecVisor http://www.cs.cmu.edu/~jfrankli/tr/franklin_secvisor_verification.pdf, UTexas Hypervisor verification http://arxiv.org/pdf/1110.4672.pdf, and application of formal methods to industrial problems like http://swtv.kaist.ac.kr/courses/cs350-08/ase08_submitted.pdf.

Beyond seL4, though, there are other efforts that NICTA incubates under the banner of Trustworthy Systems, as best described in http://www.nicta.com.au/pub?doc=4163. One of the authors of the latter paper in Leonid Ryzhyk, and in 4.2 the paper references work on the long goal of device driver synthesis, or correct-by-construction for this class of system software.

And it is the holy grail of 'correct-by-construction' for systems code that I want to mention next in a little more detail. Intel recently published a paper Device Driver Synthesis http://noggin.intel.com/content/device-driver-synthesis in the Intel Technology Journal, Volume 17, Issue 2, December 2013 titled Simics Unleashed - Applications of Virtual Platforms http://www.intel.com/content/www/us/en/research/intel-technology-journal/2013-volume-17-issue-02-intel-technology-journal.html that goes into some detail on a real instance of code synthesis.

Regarding driver synthesis, an overview of the effort may best be described in a picture, such as


above. The idea entails taking a model of the hardware to be managed by a driver plus a formal interface of how the driver interacts with the system software environment, and then synthesize the reactive code for the driver. The ideal would be automation that simply emits code, but given the human aspect of software development, such as maintenance, review, evolution, the process can act as an interactive session to have the user add code as part of synthesis, and ensure those additions are correct. The effort also focuses on making the resultant code something that has seemly names and meets other psychological constraints in working with code, such as cyclomatic complexity http://en.wikipedia.org/wiki/Cyclomatic_complexity.

Within Intel, I had the pleasure of engaging with Mona Vij who has led the team in Intel labs on evolving this technology since the summer of 2012. She, along with the Intel and external university researchers, have proven valuable, innovative parties with whom to engage. You can see the elements of our collaboration via the UEFI aspects of the effort in the paper. I believe realizing a vision of this type of work-flow would complement other efforts for the UEFI community, such as http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=UEFI_Driver_Wizard.

For additional details, the Termite page http://www.ertos.nicta.com.au/research/drivers/synthesis/home.pml calls out the collaboration. More details on the engagement with Intel and the university can be found at  

From the perspective of evolving my skills as a technologist, the engagement offered an interesting view into another approach for system software Better living through tools. It also opened my eyes to look beyond my old friend of C code to a world of functional languages like Haskell, into DSL creation, use of Binary Decision Diagrams (BDD's), SAT solvers, hardware modeling languages like DML and SystemC, too.

The industrial advantages of functional languages, albeit Lisp and not Haskell, finds an interesting discussion in the writings of Paul Graham http://www.paulgraham.com/avg.html. I recommend reading his essays, including the book version Hackers and Painters http://www.amazon.com/Hackers-Painters-Big-Ideas-Computer/dp/1449389554.

The above paper will give you a feel for the effort, but if you are hungry for more details on the underlying mechanics, I recommend visiting http://www.ssrg.nicta.com.au/projects/TS/drivers/synthesis/, too.

So again, these are my thoughts and not a plan-of-record of my employer, as my obligatory blog bio reminds people. But what I did want to do with this post is enjoin system software engineers in a conversation to think differently about how we write specifications and the process by which we refine these specifications to code, ensure that the code matches the specifications, and finally, evolve code + spec over the life of our technologies.


PS
September 2014 update.  Termite is now open source https://github.com/termite2 

Saturday, October 19, 2013

Configuring an IPV6 network boot

Earlier blogs have described the UEFI stack and network booting. This entry will talk about configuration of the boot environment.

Specifically, how do you configure a server to provide a netboot6-based image?  SUSE has written a helpful document on configuring a Linux server to support this usage at http://www.novell.com/docrep/2012/12/sles_11_sp2_for_uefi_client_best_practices_white_paper.pdf.

Recall that Netboot6 is a combination of the wire protocol defined in both RFC 5970 http://tools.ietf.org/html/rfc5970 and chapter 21.3.1 of the Unified Extensible Firmware Interface 2.4 specification http://www.uefi.org. The UEFI client machine uses DHCP as a control channel to expose its machine type and other parameters as it attempts to initiate a network boot. This is referred to as 'client initiated' network boot, as opposed to 'server initiated.' Examples of the latter include Intel(R) Active Management Technology (AMT) Integrated Disk Electronics Redirection (IDE-R), or exposing the local hardware network disk interface to the management console for purposes of the management control provisioning a disk image http://software.intel.com/sites/manageability/AMT_Implementation_and_Reference_Guide/default.htm?turl=WordDocuments%2Fsetsoliderandotherbootoptions.htm. An implementation of Netboot6 can be found at https://svn.code.sf.net/p/edk2/code/trunk/edk2/NetworkPkg/UefiPxeBcDxe/ in order to demonstrate a client-initiated download.

For client-initiated network bootstrap art like Netboot6, what are the details of the parameters?  The most important parameter entails the architecture type of the .efi image that the boot server needs to provide. The client machine that has initiated the network boot needs to expose its execution mode to the boot server so that the appropriate boot image can be returned. Recall that UEFI supports EBC, Itanium, ARM 32, ARM 64, Intel 32-bit, and Intel 64-bit. This list may grow over time with corresponding updates to the UEFI Specification of machine bindings.  Beyond a UEFI-style boot, some of my co-authors on 5970 worked for IBM and wanted to network boot a system software image over 1) HTTP and 2) not based upon UEFI technology. As such, the parameters at http://www.iana.org/assignments/dhcpv6-parameters/dhcpv6-parameters.xml cover both UEFI and non-UEFI, with the latter class including PC/AT BIOS and both PowerPC Open Firmware and Power PC ePAPR, respectively.

So RFC 5970 can be used in scenarios beyond Netboot6's TFTP-based download. This is enabled by the architecture type field extensibility, and also by the fact that the boot image is described by a URI, not a simple name with an implied download wire application protocol of TFTP as found in PXE2.1 IPV4 usages.

A way to explain this further can be done by examining our Linux configuration use case. In Linux, the DHCP server actions are performed by the dhcpd, or "Domain Host Controller Protocol Daemon." The daemon is parameterized by the file dhcpd.conf.

Within dhcpd.conf we enable Netboot6 by way of the following lines:

option dhcp6.client-arch-type code 61 = array of unsigned integer 16;

if option dhcp6.client-arch-type = 00:07 {
  option dhcp6.bootfile-url "tftp://[fc00:ba49:1625:fb0f::137]/bootx64.efi";
} else {
  option dhcp6.bootfile-url "tftp://[fc00:ba49:1625:fb0f::137]/bootia32.efi";
}

The notable aspects are 'arch type' field and then the 'tftp' term. The bootx64.efi or bootia32.efi program, also known as the Network Boot Program (NBP), when executed on the local client (hopefully with UEFI Secure Boot logic applied prior to passing control into the image) can use any of the UEFI networking API's in the protocols defined in the UEFI Spec to download further .efi images, data files, or the operating system kernel. The device path protocol on the loaded image protocol of the NBP can be used by the NBP code's implementation to find the network address of the boot server from which the NBP was loaded, too.

As mentioned earlier, this technology isn't limited to a UEFI style boot, though. A Linux PowerPC Open Firmware boot could be done with the same dhcp.conf by adding

if option dhcp6.client-arch-type = 00:0c {
  option dhcp6.bootfile-url "http://[fc00:ba49:1625:fb0f::137]/linux-powerpc-kernel.bin";
}

to enable booting a PowerPC based native binary of Linux from a web server.

If you want to take advantage of the exciting world of network boot and have a new architecture type, let me know since I'm the expert reviewer who provides the IETF with additional types, too.

Processor Architecture Types

Registration Procedure(s)
Expert Review
Expert(s)
Vincent Zimmer
Reference
[RFC5970]



That's all for today. My Saturday blogging time budget is up. Back to work.



Saturday, September 28, 2013

Where do I 'sign' up?

This post spends a little time discussing how to get pre-OS independent software vendor (ISV) content ‘signed’/’enrolled’ onto a UEFI Secure Boot protected platform.

As a quick background:
Intel (R) Boot Guard binds the OEM low level boot firmware (PI code as exemplified by SEC/PEI/DXE) with the hardware, so the Boot guard trust anchors would not directly interface with the trust anchors for 3rd party UEFI content.  Details on Intel Boot Guard can be found on page 4 of http://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/4th-gen-core-family-mobile-brief.pdf.  

People often ask about the relationship of something like Intel (R) Boot Guard and its "Verified Boot" versus UEFI Secure Boot, as defined in chapter 26 of the UEFI 2.4 specification. We talked a little about this earlier, too, at http://uefidk.intel.com/sites/default/files/resources/Platform_Security_Review_Intel_Cisco_White_Paper.pdf, page 16.  The “Reset Time Verified Launch” in Figure 5 logically maps to something like Intel(R) Boot Guard. The verification happens 'before' UEFI PI code and vets the provenance of that code, typically if the code was created and updated under the authority of the system board manufacturer. UEFI, on the other hand, is on the right hand side of that flow.

In other words, the underlying PI-code update key, say for validating a capsule update (install time verification) or the embedded signature of the PI code (load time verification) should not be the PK but some other system board vendor-managed key store.  Recall that on certain x86 systems the end user could even edit the PK via a physically-present setup page.  In that latter case, having the end user control the PI update key (and associated system firmware updates) is often not desired.  In the PI specification there are definitions of signed firmware files and volumes, but there is no defined policy store and trust anchors for 'Secure Boot' of PEI and DXE elements. 

In the end, users want end-to-end integrity, though, so both protection of the underlying firmware and the run time are important.  This was discussed a bit in the Intel Developer Forum presentation earlier in the month with the system architecture picture updated to the revised below.

Note in this picture above that Intel (R) Device Protection Technology with Boot Guard surfaces from the system hardware and precedes execution of the PI SEC/PEI/DXE codes.

UEFI Secure Boot, on the other hand, is intended for 3rd party UEFI content, such as UEFI drivers or applications on the UEFI system partition.  Intel(R) Boot Guard and PI code verification keys should have their own manifest and storage structure.  For the 3rd party trust anchors, the place where this enrollment would happen is with the UEFI Secure Boot key hierarchy.  The hierarchy for UEFI Secure boot includes the PK, KEK, DB, DBX.  The factory-default configuration typically entails a PK that is owned by the OEM, and the PK authorizes updates to the KEK.  The KEK is OS Vendor1 + OEM + other OS vendors, and the KEK entries authorize updates to the DB/DBX.  DB is the ‘allowed’ list of code that can execute, and for a Microsoft (R) Windows8 machine contains a Microsoft OS certificate, the Microsoft UEFI CA cert, and possibly other OSV/ISV entries. 

Now for going from theory-to-practice-

Given a population of UEFI Secure Boot capable machines in the field, how is a pre-OS Independent Software Vendor (ISV) able to deploy content (i.e., the action item from above)?   The short answer is that the ISV has 2 options:

-          1. Sign up w/ Winqual and get the UEFI driver/application signed by the UEFI CA

and/or
-          2. Create own verification certificate and

o   Have end user enroll manually

and/or
o   Have OEM preinstall (or update in field via firmware update)


An ISV can do 1+2 above since UEFI Authenticode-based executables support ‘multisigning’ so that they can be signed by BOTH the UEFI CA and the ISV’s own key (see more on the final links below w/ SUSE example).

For the first option 1. above, the ISV can sign up w/ Microsoft Winqual and submit their content to be signed by the Microsoft UEFI CA.  Most ISV's, IHV's, and non-MSFT OSV's already has a Winqual account if they deliver signed Windows drivers today since Microsoft has been doing kernel mode driver signing since Vista SP1.  In addition, most IA machines that support UEFI2.3.1 Secure Boot carry a Microsoft UEFI CA DB certificate, so getting signed by the MSFT UEFI CA will mean that the ISV's .efi UEFI driver or application will simply work on a large class of PCs.  More info on the process can be found at:

For the second option 2. above, if the ISV wishes to generate its own roots and manually enroll in a PC (e.g., using PC setup screens) or distribute its keys for the OEM’s to pre-enroll, some details on the process can be found at http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=SecurityPkg

Beyond this information, some of the practices of a Security ISV like McAfee for their pre-OS UEFI content can be found at

If you have a machine at home, you can use some of the flows described in the white paper above on running dmpstore and other commands at the UEFI shell to discover the configuration of UEFI Secure boot.  A more user friendly way is to run the Secure Boot Checkup Utility http://apps.insyde.com/sbutil.html from Insyde on your Microsoft (R) Windows 8 machine.  Thanks to Jeff Bobzin at Insyde for pointing this tool out to me.

This is the report from my Asus Windows 8 Intel (R) i3 touch laptop.  Well, my former laptop prior to my fourteen year old daughter commandeering it.  The output from the report proceeds below-

Secure Boot Status on this system:
System Status: MS Required KEK: MS Required OS Cert: 3rd Party (MS CA):
Secure Boot Enabled Present Present Present

UEFI Variables:
SetupMode: 
SecureBoot: 
OsIndicationsSupported: 

BootOrder Item List: 
BootCurrent: Boot00000 1 0000000000000001 0000 0000 Windows Boot Manager

Secure Boot Database Contents:

PK Variable Certificate (Platform Master Key):
X.509 Certificate:
CN=ASUSTeK Notebook PK Certificate

KEK Variable Certificates (Database Management):
X.509 Certificate: X.509 Certificate: X.509 Certificate:
CN=ASUSTeK Notebook KEK Certificate 
CN=Microsoft Corporation KEK CA 2011 
CN=Canonical Ltd. Master Certificate Authority

db Variable Certificates and Hashes (Allowed Signers):
X.509 Certificate: X.509 Certificate: X.509 Certificate: X.509 Certificate: X.509 Certificate:
CN=ASUSTeK Notebook SW Key Certificate 
CN=ASUSTeK MotherBoard SW Key Certificate 
CN=Microsoft Corporation UEFI CA 2011 
CN=Microsoft Windows Production PCA 2011 
CN=Canonical Ltd. Master Certificate Authority

dbx Variable Certificates and Hashes (Forbidden Signers):
X.509 Certificate:
CN=DO NOT TRUST - Lost Certificate

The interesting thing about this machine is that there is both a ASUSTeK KEK and Canonical KEK, along with the Microsoft KEK.  So this set of KEK entries includes one for the OEM and two alternative operating system vendors, namely Microsoft for Windows and Canonical for Ubuntu http://www.ubuntu.com/.

Below is a friendlier view of the tool in action.



Finally, back to closing thoughts on the Intel Developer Forum. 

Within that deck, one of the slides I enjoyed the most was the quote on slide 21 from SUSE that read:

"UEFI Secure Boot no longer an issue to the Linux* World"


Regarding IDF, it was great having the opportunity to meet with people from all across the industry.  I found the perspective of what I do from the different companies refreshing and helpful in informing customer and business-focused activities with my colleagues going forward.  Translation: I learned about a lot of things I need to get done quickly. 

This IDF talk marked ten years since my first IDF presentation in San Francisco.  I shuffled a bit less than the first talk, but I have to admit that I haven't evolved to the aplomb and sophistication of a Ted Talk http://www.ted.com.  The irony is that one of those talks I delivered in 2003 read on porting the Intel Framework (PEI/DXE) for different platforms, factoring silicon initialization code, etc. Since 2003 Framework 0.9x specifications became PI1.3, the EFI Developer Kit (EDK) became EDK2, and the preceding UEFI track talk at IDF 2013 discussed factoring all of the silicon initialization PEI modules as this new Firmware Support Package (FSP) http://www.intel.com/content/www/us/en/intelligent-systems/intel-firmware-support-package/intel-fsp-overview.html.  Oh yeah, Windows 8 shipped with UEFI as the default boot loader option recently, too.  Quite the decade indeed.

Speaking of getting things done, I had better end today's entry and get back to work.

Cheers

2/15/2015 update -
My friend's book http://www.apress.com/9781430265719 has a good description of Boot Guard in chapter 6.  Take a look.

Sunday, September 8, 2013

End of summer '13

The unusually warm summer proved a bit confusing to the psyche.  The return of the Pacific Northwest rain of late brings me back to normalcy.

I haven't blogged in a while, but I must recommend and continue to be impressed by he writing of Tim Lewis at http://uefi.blogspot.com/.  Really good stuff.  I have appreciated the opportunity to work with Tim over the last decade or so and I am in continual awe of his intelligence and prolific output.

Let's begin with something of a meta-blog as an introduction today.  I sometimes censor my content for fear of being read as an advocate of a specific technology, or detractor of another. Sometimes 'is' versus 'ought' gets confused in such writings, namely the difference between describing a technology and evangelizing the same.  I felt alone in this tension until listening to Violet Blue and her description of today's confusion around 'information' versus 'advocacy' with the tech culture. The talk 'Hackers As A High-Risk Population [29c3]' https://www.youtube.com/watch?v=zq-bloM4Cmo elaborates on this point, among others.  

Of course, I surely have subconscious confirmation bias based upon areas with which I'm more familiar, but information comprises the intent of the following.   

Enough of the caveats, let's talk about architecture versus implementation.   

In the world of central processing units and hardware, we often refer to the distinction as 'architecture' versus 'micro-architecture.' The former describes the public description of the instruction set architecture (ISA), memory model, protection modes, and other modalities of the hardware that provide the consistency against which compiler writers, application developers and system software engineers can base their software development.  For Intel, these details are codified in the Software Developer Manual (SDM) http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html.  The micro-architecture, on the other hand, provides details on 'how' the architecture is implemented given adherence to the architecture as an outward constraint, but compounded with additional constraints of design inventiveness, power envelopes, transistor budgets, design invention, and schedule constraints, among others. It is in the world of micro-architecture where techniques like in-order versus out-of-order, additional functional units, bus width, number of ports to the memory and register file, micro-coded versus hardwired, and other techniques come into play. 

That's for the world of CPU's and hardware.  Correspondingly, the world of software defines the demarcation as 'interface' versus 'implementation.'  The former include the POSX API's, Win32, the UEFI Boot Services, and to even stretch the metaphor a bit, network wire protocols.  For the latter, implements include the Linux kernel, Microsoft Windows, the edk2 development project, and the venerable BSD TCP/IP network stack.  The line is not always clear initially, and some implementations become a de facto implementation, such as the Microsoft DOS binary executable.  The same holds true for the original PC/AT, where the public description http://bitsavers.trailing-edge.com/pdf/ibm/pc/at/1502494_PC_AT_Technical_Reference_Mar84.pdf of the hardware and software 'implementation' became the 'architecture' foundation, now referred to nostalgically or disparagingly as 'legacy', of the PC ecosystem.

Let's get into some details on where the architecture and implementation come together with respect to host firmware.  From a most generic taxonomy, there are two phases of execution.  The first is a 'platform initialization' and the second is a 'boot loader' phase.  The former phase entails initialization of the system board hardware and passing control to the boot loader phase.

All of these processes of booting entail passing control from one domain of execution into another.  These can include system boards that need boot firmware to launch a shrink wrap OS or a vertically integrated embedded system wherein the firmware producer and the OS producer are the same entity.  There are often different instances in the market depending upon the requirements of the business entities.

For the first phase, here is a quick chart mapping two well-known boot firmware technologies found on Intel Architecture.

Platform Initilization     coreboot          UEFI PI
Reset vector               Boot block           SEC
Memory init                ROM stage          PEI
I/O init                        RAM stage          DXE
Boot loader phase       Payload               BDS

Of the boot loader phases using coreboot payloads, these can include U-Boot as an implementation with SeaBIOS and a direct kernel load as alternate 'hand-offs' of control. Those hand-offs of control can include the Advanced Configuration and Power Interface (ACPI) http://www.acpi.info or the flattened device tree (FDT) evolved from open firmware IEEE 1275 http://www.openfirmware.org/1275/.  For the boot loader phase of the UEFI PI, the most well known embodiment is the Unified Extensible Firmware Interface (UEFI) presently at revision 2.4 http://www.uefi.org.  

For an architecture to implementation mapping, SEC, PEI, DXE, and BDS can be found in the 5 volumes of the UEFI Platform Initialization (PI) specification, presently at revision 1.3, at http://www.uefi.org.  The architecture is represented by .h files in the https://svn.code.sf.net/p/edk2/code/trunk/edk2/MdePkg/ and implementations of the platform portable elements in https://svn.code.sf.net/p/edk2/code/trunk/edk2/MdeModulePkg/.  For coreboot, an overview can be found at http://www.coreboot.org/Welcome_to_coreboot and representative implementation practice at http://review.coreboot.org/#/q/status:open,n,z.  

The story is not so straightforward, though.  Coreboot can have edk2-style firmware as a payload, such as found in the 'PIANO' project, or 'Payload using Tiano.'  Recall that "Tiano" was Intel's original code-name for what became the Intel (R) Framework and EFI, then UEFI PI and UEFI reference implementation.   You can still find reference to 'Tiano' on the edk2 project http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=EDK2, too.  Correspondingly, the Boot Device Selection (BDS) driver in a edk2-based PI implementation can do a direct Linux kernel load and publish a FDT, as found in ARM Ltd's (R) adaptation of edk2 in https://svn.code.sf.net/p/edk2/code/trunk/edk2/EmbeddedPkg/.  

The point of FDT/ACPI is to expose non-discoverable resources to the operating system.   And the point of U-Boot/UEFI is to assist in loading the operating system kernel via the local NV store, disk, or network and passing control to the same.  We discuss some of the uses of UEFI/PI for embedded at http://linuxgizmos.com/using-uefi-in-embedded-and-mobile-devices/, too.

Recall my mention of Intel Architecture above.  In general, UEFI has processor bindings to 32-bit or IA32 and 64-bit Intel Architecture, or 'x64' as known in the UEFI spec.  The latter has been alternately called x86-64, AMD64, EM64T, and Intel64.  UEFI also supports Intel(R) Itanium and both 32-bit and 64-bit ARM ISA.  The latter 2 are Aarch32 and Aarch64, resp.  Beyond that, there have been ports to other architectures not covered by the UEFI specification, such as 32-bit MIPS http://sourceforge.net/projects/efi-mips/.

Correspondingly, the coreboot tip has IA32 support and recently has added 32-bit ARM.  In the past, PowerPC support was also found in this project.

U-Boot has an even broader CPU support set since for certain CPU and platform architecture, the hand-off to the "Boot Loader Phase" can be done by opaque firmware and hardware in the platform.  In other words, the "reset vector", "memory init," and "I/O init" happen in the 'hardware' and do not require UEFI PI or coreboot style processing.

So in the spirit of 'information' versus 'advocacy,' you can see that the different facets of the solutions above can map to alternate deployment needs.  For a shrink-wrap OS that wants a guarantee of system board behavior into the "boot loader phase," the industry standard ACPI+UEFI couplet has some merit. For defacto standards like the kernel load protocol of Linux, FDT+U-Boot or FDT+UEFI may suffice.  

And the distinction between architecture and implementation rears its head again with respect to security.  As noted as recently as Blackhat this year by Yuriy Bulygin, et al. http://www.blackhat.com/us-13/speakers/Yuriy-Bulygin.html in their presentation http://c7zero.info/stuff/Windows8SecureBoot_Bulygin-Furtak-Bazhniuk_BHUSA2013.pdf, any flaws in implementation of the architecture can allow for bypassing of the controls for which the architecture was intended to enforce. In this case, the policy objects and implementation of UEFI Secure Boot represented the implementation artifacts that failed to successfully refine the architecture to the code in the system board storage. This is an example of the classic assurance consideration and reads into the firmware design space.  Security is not alone.  Performance, manageability ease-of-use, size, and other metrics can be impacted by the refinement process of a high level architecture to code.

As a final thought, I will be at the Intel Developer Forum next week talking about "UEFI Secure Boot and Linuxhttps://intel.activeevents.com/sf13/connect/sessionDetail.ww?SESSION_ID=1064.  This talk reads on many of the themes treated in this blog around building an implementation of an architectural feature, specifically the administration of policy objects for features like UEFI Secure Boot. For this talk, the architecture of UEFI Secure Boot can be found in Chapter 26 of the UEFI 2.4 specification.  This has our familiar friends the Platform Key (PK), Key Exchange Key (KEK) and the allow/disallowed list of the db/dbx.  These authenticated variables and how they inform the policy-based load of 3rd party UEFI images is well defined in that document.  What is absent from the document is how a security administrator (e.g., security_admin_r) or end user manages the UEFI Secure Boot behavior.  This gap stems from the absence of manageability interfaces to a given platform in the UEFI Specification, among other things.

So in order to appreciate UEFI Secure Boot as an end user, the description and implementation of the feature in chapter 26 is necessary but not sufficient.  Even for assessing the overall integrity goals of the feature, you need to assess the end-to-end solution, such as shown below in one embodiment.

End-to-end platform integrity


As you can see from above, the underlying UEFI PI implementation must guarantee its provenance, such as through NIST 800-147 http://csrc.nist.gov/publications/nistpubs/800-147/NIST-SP800-147-April2011.pdf specified updates.  Another means by which to effect the same would be a load-time verification of the UEFI PI code from the 'hardware' block that I will discuss at IDF.  Finally, SUSE will discuss some options to support the "OS Secure Boot."  Note that the UEFI specification only reads on the center of the diagram, and even for this portion, it does not describe how to manage the PK, KEK, and DB/DBX other than through cryptographically signed updates of the same by the private key owner.

For the latter key management, a end user who is a kernel hacker may manage the key store directly via physically present interaction with the firmware setup screens and load his/her own keys. A consumer, on the other hand, just wants it 'to work' with live CD's and other shrink-wrap binary images from different OS vendors without any administrative changes. In the former two cases, the 'owner' and 'user' are typically one in the same. A corporate IT, on the other hand, only wants the IT staff to manage and change these options since the 'user' or knowledge worker of a client machine is distinct from the 'owner', who is typically the knowledge worker's employer.  And the IT staff simple acts as an agent of the owner and the owner's elevated privileges.

On the subject of assurance, many people use UEFI variables for storing PI-specific content. The problem with that scheme is that after the signalling of EFI_END_OF_DXE_EVENT_GUID from volume 2 of the UEFI PI spec at www.uefi.org (aka "Exit PM_AUTH" from http://www.intel.com/content/dam/doc/white-paper/uefi-pi-tcg-firmware-white-paper.pdf or "Exit Mfgr Auth" from http://www.uefi.org/sites/default/files/resources/Intel-UEFI-ThreatModel.pdf) and/or when BDS does a connect on the driver load list and the UEFI boot variables, 3rd party UEFI content will run. Even in the case of UEFI Secure Boot, you know the provenance of code but not if the UEFI loader or OS runtime will attack the platform. Given that case, the PI-only variables should be prohibited from change by the 3rd party UEFI drivers, applications and operating system runtime.  One way to do this protection is to use the edk2 variable lock protocol (EDKII_VARIABLE_LOCK_PROTOCOL) http://permalink.gmane.org/gmane.comp.bios.tianocore.scm/842.  Note that this API is in the edk2 project but not prefixed with "PI" or "EFI" since it does not exist in the industry standard specification. This API allows for downgrading a variable to read-only (RO) from read-write (RW) prior to running third party code. It also covers a gap in the UEFI  and PI specifications wherein the SetVariable API in chapter 7 of the UEFI specification does not read on creation of RO variables; the main UEFI specification only describes the need for a set of RO variables to exist.  How the underlying PI-based code creates these RO UEFI variables was left as an exercise for implementations.  The EDKII_VARIABLE_LOCK_PROTOCOL provides one 'answer' to that exercise.

So what have we seen in this blog?
1) Explanation of technology without marketing for same
2) Many examples of architecture versus implementation
3) For boot firmware, there are many different architecture and implementation
4) Take due care in traversing from architecture to implementation
5) Take a systems view of a solution beyond a given technology's architecture and implementation




Saturday, March 16, 2013

A Technical Career Path

I was recently asked to provide a short video on technical leadership and the technical career path.  When I asked the opinion of some friends on the subject matter, one response was "tell them it's 'better to be lucky than good.'"  I was a bit too tired in that AM to reply with something witty like Einsteins' quote "God does not play dice." or some such, but I do believe that there are some guidelines that can help in a technical career.   To that end, I have broken those guidelines out into a few key points.

To begin, for any hope of success in a technical career path, you must have....

1. Passion

The modern work routine doesn't fit nicely into an 8-5, M-F schedule.  Whether it's meetings with colleagues in other countries or thinking about a particular problem while in the shower, the tasks will remain with you.  So if you are not passionate, excited, and engaged on the domain of application for which you are employed, this constancy of the job will become a torture and a prison.   Instead, you should be excited about the subject and eager to continue exploration in the space.

Of course, passion is laudable but it will not necessarily earn you a paycheck.   Upon the foundation of passion you need to build a layer of....

2. Technical IQ

By technical IQ I mean cultivating domain expertise in an area that includes both the theory of and ability to implement the domain.  In fact, in the software world, you can alternately get paid for having some deep equity in domain expertise, with executable results including writing specifications, engaging customers on requirements, etc, or the implementation skills within the confines of a given software language, such as C, C++, Haskell, Python, etc.   The domain expert can talk a lot but often not deliver the product, such as the classic 'architect.'  The software engineer can reduce the architect's specification into a programming language embodiment.  You can get paid for either, but you can get paid a lot more for being able to do both.

In addition to domain expertise, I often argue for the 'broad and deep' approach.  One way to guide oneself in the breadth is to look at adjacency's.  For example, doing boot firmware like UEFI, I need to understand the layer above and below.  Above includes the operating system UEFI boots and interact with at run time.  The Below includes the hardware that the UEFI PI layer initializes on restarts and helps manage at run time.  The depth is in UEFI, PI, ACPI, C, assembly, networking protocols, cryptography, CPU and platform macro-architecture.   The breadth includes Windows, Linux, RTOS's (above) and micro-architecture, PCI, USB, I2C, GPIO, SD, TPM, SRAM, DRAM, DDR, digital and analog circuit design below.

Since I want to narrow my efforts to the most appropriate for my employer, the breadth also includes the business environment, strategy, and market goals within which I exercise by domain expertise and implementation skills.

Technical IQ is great, but the days of the lone programmer and inventor are long past.  The passion and technical IQ need to be done exercising....

3.  Emotional IQ

By Emotional IQ I mean the ability to work with, for, and through people.  Any corporation or technical endeavor invariably entails interaction with people.  In fact, I would argue that business entails concerns of people ahead of the technology details.  And to that end, the sophistication of this engagement can be scoped better.  As I remind myself, there doesn't exist a "Moore's Law" for human psychology.  Follow the golden rules, respect others, find value and richness in interacting with others, and pursue those opportunities beneficial to all.   As you advance in your career, you begin to rely on others as much as others will rely on you.  Just as you should always seek out mentors in your career, progressively move into the role of mentor for others.

So now that you have the passion, or Aristotle's Prime Mover that starts you on the journey.  This passion is coupled with both formal/informal education and on the job experience for Technical IQ.  And the Technical IQ is exercised in the context of others using your Emotional IQ or "EQ."   So where do you go next.   With these three elements in hand, .....

4.  Do something wonderful

You have the passion for, skills to, and colleagues with whom you can be a technical leader, so lead.  Take a risk.  Do something wonderful.  All technology was someone's idea at some point.   In fact, as Alan Kay says, the 'best way to predict the future is to invent it.'   I would argue that the landscape for invention, innovation, and creation has never been more fertile.   The lines between the hardware, firmware, operating system, application, and middle-ware are constantly moving and blurring.  Be that agent of change.

So now that I have 1-4, is there more?   Yes, exercise 1-4 with the following in mind....

Philosophical Postscript

With all of that being said, the ideals and reality often don't meet for some.  In fact, I'm often asked 'why can't I get promoted' or other such concerns.   My reply is pretty simple:  "It's the journey, not the destination."   Notice that nowhere in stages 1-4 above did I have a 'become chief super scientist' or other such exalted title.   The title doesn't matter.   What matters are the stages above.   And in fact, I'd even mention something like Zen and the art of archery - the more you try to hit a certain target (e.g., 'promotion'), the harder it will become.   Treat work as play.  Enjoy the trip.  Feed your passion (1) for a subject by learning more about it (2) and engaging with others on the subject (3), culminating in creation that helps the business (4).   Being diligent on these levels will yield the appropriate fruits.


Sunday, February 24, 2013

Anniversary Day.Next, Arch P*'s, and some stack history

Today makes 16 years at Intel.  Sunday is usually my catch-up day for work tasks or extra credit work (e.g., patent drafting), but given that I posted an entry on this day last year, I'll steal a few minutes to post something today.  If I keep my wits about me maybe this can become a tradition?  Induction will argue that it might be so if I hit the 'publish' button before the end of the day.  0, 1, ... infinity, right?  OK.  Enough of that.   Popper and Hume might rise from their graves and smite me for induction invective if I go on that path.

The first topic I wanted to touch upon today includes the intent behind the architectural p*'s in the UEFI Platform Initialization (PI) Specifications.  Namely the architectural PEIM-to-PEIM interfaces (PPI's) and Architectural Protocols (AP's) in Volumes 1 and 2, respectively.  I have been meaning to cover this topic for a while, but today's posting is motivated by James B's foils @ http://blog.hansenpartnership.com/wp-uploads/2013/02/UEFI-Secure-Boot-2013.pdf, namely slide 15.  In this presentation deck, not 'paper' as some people are wont to describe foils these days, James discusses overloading the security protocol from the PI specification from his UEFI application.   As James' loader is a pure UEFI application, any dependency upon an underlying PI interface breaks portability.  There is no reason that UEFI interfaces need be built upon a PI-based underlying implementation, for example.

We mention this in many places, including page 12 of UEFI_Networking_and_Pre-OS_Security or http://noggin.intel.com/technology-journal/2011/151/uefi-today-bootstrapping-continuum, viz., "PI, on the other hand, should be largely opaque to the pre-OS boot devices, operating systems, and their loaders since it covers many software aspects of platform construction that are irrelevant to those consumers."

The history of the PPI's and AP's in Intel Framework, and subsequently the UEFI PI specifications, was to abstract a portable PEI and DXE core from the respective platform.   The sources for the cores are intended to be kept platform neutral such that they can be seamlessly compiled to any target architecture, which today includes IA32, x64, Itanium, and 32-bit ARM for the edk2 at http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=EDK2.  As such, the arch PPI's and AP's provide a hardware or platform abstraction lay (H/PAL) to the cores.  Back to James' foils above, he took advantage of a design choice where the protocol's were left in the UEFI protocol database after the DXE core had bound itself to those call-points.  A conformant PI implementation of UEFI could have uninstalled that protocols prior to entering BDS and invoking 3rd party UEFI drivers and applications, for example.

Omission of the AP's from the database and precluding usage by 3rd parties is not just hygiene.  Abuse of the timer AP, for example, by an errant or malevolent UEFI application could change the timer tick in a way that was not consistent with the DXE core's original programming of the time base via the timer AP.

So the lesson from above includes (1) if you want to be a portable UEFI driver and application, don't invoke UEFI PI AP's, and (2) as a platform creator, be wary of the protocols you leave installed in the database prior to invoking 3rd party UEFI content.

Next on the list today includes a short history of the UEFI networking stack.  The original EFI networking stack was part of the EFI1.02 specification back in 1999.  We modeled the network stack on the PC/AT networking support, including the PXE base code (BC) and UNDI interfaces.  The latter 2 API's were defined in the PXE2.1 specification.  The PXE (pre-boot execution environment) specification contains both a wire protocol for network boot and a set of API's that a network boot program (NBP) uses so that when it is downloaded onto a PC/AT BIOS client machine it can continue usage of the client machine's network stack.   For EFI1.02, we adapted the PXE BC and UNDI API's into EFI equivalents, while preserving the network wire protocol aspects, such as the DHCP control channel, TFTP-based file download, etc.  EFI added an additional software API on top of UNDI called the Simple Network Protocol (SNP), too.  So from BIOS to EFI we preserved PXE BC, UNDI, and added a SNP between the former two.

After EFI1.10 and its inclusion of EBC and the EFI Driver Model, we thought of what a "EFI1.2" specification might entail.  One aspect of the pre-OS we explored included more efficient networks.  The problem we discovered was that SNP could only be opened by one agent, namely the PXE BC.  When other network drivers were added, such as the circa-1999 BSD TCP/IP port to EFI, we had to unload the PXE BC in order to have the TCP/IP stack active.   Against this background of a single SNP consumer, we designed the Managed Network Protocol (MNP).   MNP provides for multiple listeners and also provides a non-blocking interface, unlike the single consumer, blocking nature of SNP and UNDI.  We took the concept of multiple consumers and non-blocking for the higher layer protocols, including UDP, TFTP, DHCP, and TCP/IP.   In doing so, we were able to rewrite the PXE BC application to be a small consumer of the below-listed services versus the monolithic stack embedded in the original PXE BC.

Since EFI1.2 never came to light, MNP and the rest of the modular stack definition were contributed into the UEFI2.0 specification as part of the UEFI Forum's effort.  This was timely in that ISCSI was added to the UEFI2.0 corpus, and implementing ISCSI and PXE on the same modular stack proved to be feasible.   The next question posed to the pre-OS networking stack was what to do with PXE.  Recall that the EFI, and now UEFI, interfaces to PXE included the PXE 2.1 wire protocol.  We were faced with evolving PXE to an alternate wire protocol or augmenting PXE and the rest of the networking stack to meet the emergent needs in the industry, such as internet protocol version 6 (IPV6).   Given the extant infrastructure of PXE servers and associated scenarios, included blended scenarios such as 'PXE boot the NBP OS installer, mount an ISCSI share, install into the ISCSI block device', etc, we opted to draft PXE into IPV6 with parallel capabilities.   We coined the effort to have network boot on IPV6 'netboot6' and codified the elements of this capability in RFC 5970 http://tools.ietf.org/pdf/rfc5970.pdf and chapter 21 of the UEFI 2.3.1c specification.  5970 was evolved as part of the DHC WG in the IETF and describes a broad class of network boot transports described by a URL, which includes HTTP, TFTP, ISCSI, NFS, etc.  Netboot6 opted for the TFTP-based transport to have a parallel flow to the PXE 2.1 wire protocol, but there is no reason going forward that alternate boot transports can be used.

Seeing is believing, though, so you can find the networking infrastructure mentioned above at http://edk2.svn.sourceforge.net/viewvc/edk2/trunk/edk2/NetworkPkg/.

I regret that this blog and many of the white papers referenced in the past have not treated the UEFI networking stack with much detail.  I'm working to get some more collateral out in the open, especially given how to compose pre-OS networking and fast-boot.  One piece of wisdom that I want to mention is that the UEFI system should not attempt to start any of the networking stack elements unless there is a boot target that includes PXE or other networking-facing service.  This is in the spirit of the UEFI driver model where you should not connect a driver unless needed to boot.  This sparse connect is how UEFI achieves its fast boot times relative to PC/AT BIOS's that had to spin up all of their spindles and network interfaces in absence of having a stylized boot object like the UEFI boot targets with their device paths.  Given that IEEE 802.3 Ethernet devices can take 7 to 20 seconds to effect cable detect actions, any touch to the network will kill chances of achieving the 2-3 second boot times expected by modern operating systems.

Other advice I will include on network boot includes policy decisions of orchestrating IPV4 versus IPV6 on network accesses, etc.  So beyond having a more detailed overview of the stack, fast boot and network selection policy, let me know if there are additional topics on the network stack that you would like to see treated.

Well, that's it for anniversary year 16 blog.  Let's hope that the inductive argument holds true and I am creating the year 17 variant of this blog.

Cheers

Sunday, February 10, 2013

What is 'Compatibility'?

So why another blog just a week after the last one?   Given the aperiodic nature of my earlier blogs, this frequency seems a bit out of character.  Well, I finished reading Fowler's Passionate Programmer and he exhorts the reader to practice, practice, practice.  Not just code, but writing.  And one venue for writing is a web log.  No referees, reviewers, publisher, etc between my typing and publication.  So here you go, for better or worse.

Now back to the topic of this blog.

There is often talk of 'compatibility' in the computer industry.  From the perspective of the platform firmware, we define compatibility as 'the software you choose to run.' In this taxonomy, compatibility is not some absolute term or Platonic truth living outside the world but instead very much a contextual term.

So this is all well and good, but how does this relate to the platform firmware specifically?  In the case of the boot firmware, we think of compatibility in terms of the PC/AT BIOS and its interfaces.  These interfaces include the 16-bit int-callable interfaces published by the system board BIOS and BIOS extensions for use by booted software.  The 'booted software' can include the MBR first stage loader and subsequent system software. The BIOS industry referred to these int-callable interfaces, such as int10h for video, int13h for disk, int16h for keyboard, along with data structures like the BIOS Data Area (BDA) and Extended BIOS Data Area (EBDA) as the "BIOS Runtime."  This BIOS Runtime is distinct from the code that executes after a platform restart and the germination of the BIOS runtime interfaces.  This phase of execution has been historically lumped under the term Power-On Self Test (POST).

Back in the early 1999 when the Extensible Firmware Interface (EFI) was first being deployed, this phase of execution, or the Boot Services (BS) and subsequent Run Time (RT), were really akin to the BIOS Runtime.   This parallel is borne of the fact that the EFI specification, nor its successor the UEFI specification, dictates 'how' these environments appear.  The pre-EFI/UEFI phase is opaque to and distinct from the UEFI BS and RT, just as POST is distinct from the BIOS Runtime and its int-calls.

In the case of EFI, the Intel Platform Innovation Framework (aka "Framework") and its successor, the UEFI Platform Initialization (PI) specifications, defined a stylized POST phase.  This includes the SEC, PEI, and DXE phases of execution.  But just as EFI is distinct from its preceding phase, the PI phase does not necessarily have to create a full UEFI environment.   And it is this PI versus UEFI where compatibility comes into play.   You see, as EFI and UEFI worked to gain acceptance, the bulk of platforms in the market since 1982 supported PC/AT BIOS and 16-bit BIOS option ROMs and OS loaders.

To leverage that large class of 'legacy' BIOS infrastructure, the Compatibility Support Module (CSM) was borne.  The CSM was developed by a project that preceded the "Tiano" program but was codified into a Framework Specification http://www.intel.com/content/www/us/en/architecture-and-technology/unified-extensible-firmware-interface/efi-compatibility-support-module-specification-v097.html.  This earlier program referred to the compatibility code as the 'fur ball' with the hope that one day a native mode firmware stack would prevail and 'cough up' the compatibility codes.

The CSM provides a way to encapsulate a BIOS runtime inside a Framework or PI-style firmware.  The CSM specification abstracts the information collected during a PI-style POST, namely SEC, PEI, and DXE, and passes that information into the CSM wrapped version of a BIOS 16-bit runtime.  We often refer to this binary in our edk2-based trees as a CSM16 binary.

Recall that UEFI provides services for blocks such as the EFI Block I/O protocol, and the BIOS supports similar capabilities via the int13h API.  To leverage an underlying 3rd party storage option ROM's 16-bit disk services, a UEFI driver can be generically written to thunk or call down into the 16-bit code.  The thunk is provided as a general utility via a API member of the CSM protocol.  There is also a reverse thunk, but it is rarely if ever used.  You see, a CSM16 cannot practically call back into native UEFI 32-bit or 64-bit code because when control is passed to a 16-bit MBR loader, the int15h E820h memory map that the legacy OS receives has the UEFI boot service pages mapped as available.  And since there is no equivalent of ExitBootServices() for a legacy OS boot, UEFI services in the form of boot services drivers cannot reliably be invoked via a reverse thunk after the int19h IPL.

EFI/UEFI calling down into PC/AT BIOS CSM16, on the other hand, can be done throughout the boot services phase since the CSM16 services are available until ExitBootServices, so a BiosBlockIo or BiosVideo driver can reliably thunk to the int13h and int10h CSM16, respectively.   The only way to have 32-bit or 64-bit native codes that are shareable between CSM16 and a DXE-based UEFI BS is to put the codes into something like a DXE SMM driver.   The downside of the latter is that 3rd party UEFI drivers cannot be loaded into SMRAM.  This only works for system board hardware, such as USB ports in the chipset.

And now even within EFI/UEFI there are compatibility concerns.  EFI1.02 to EFI1.10 introduced OpenProtocol, and implementations of the EFI1.02 HandleProtocol had to use the later reference-counted API to implement the earlier non-reference counted API.  Then UEFI2.0 in 2006 deprecated UGA and DeviceIo.  From UEFI2.0 to 2.1, then 2.2, 2.3, 2.3.1 have introduced new services.  Framework HII to UEFI HII.  A style of programming to maintain compatibility often includes reading the specification version in the system table in order to take the appropriate code path.  The same holds true for PI.  PI became with the donated Intel Framework specifications, and from Framework 0.9x to PI 1.0, 1.1, 1.2, 1.2.1, 1.2.1a, etc there have been similar concerns with SEC, PEI, DXE, SMM.

So that's my blog for today on compatibility.