From Brandon's Tinkerings
Jump to: navigation, search

Intro

PXE (Preboot Execution Environment) Booting, or just Network booting in general is very interesting, at least to me, and a few others. As I believe it was Marty Connor in this awesome video "gPXE: Modern FOSS Network Booting" said that some people get really excited over booting machines over networks (including the Internet!) while others... not so much.

Well, I'm one of those people who gets really excited over the idea of booting machines over a network, and I can't really put my finger on why, it's just awesome to me.

So, I wanted to document the netboot setups that I use at my home, and my work. This entry consists of my home network. My work one, I'll put in another entry (as it's significantly different in it's programming, but does the same functions), and link here.

Now, network booting isn't for everyone, and it doesn't fit every situation, so your mileage will vary greatly.

My home network consists of iPXE, PHP scripting, and separate utilities. All of this is detailed below... so lets begin!

What does this page assume?

  • You have a working network
  • You control your DHCP Server
  • You have control of your DNS server
  • You have a working webserver
  • Basic understanding of PHP
  • Have a basic understand of whats involved with PXE Booting, even if it's skimming over the Wikipedia page
  • Have a machine that is capable of picking the network card to boot from, via PXE (On most Dell systems, you need to go into the BIOS, Integrated Peripherals, and mark the NIC as "On W/ PXE", not just "On", or "On W/ ImageServer"

My Environment

  • Linux, Distro: Gentoo (~AMD64 "Unstable")
  • Apache 2.4.3
  • PHP 5.4.7
  • tftp-hpa 5.2
  • iPXE (current GIT master)
  • Misc Utilities like Drive Fitness Test, SeaTools, Memtest, etc.

The Basic Process

-- My Setup

  1. Computer powers on, and selects the NIC to boot from, either via interaction, or it being the first device
  2. The native PXE Stack (iPXE (flashed onto the ROM/BIOS), Intel, Broadom, Realtek, etc) brings up the network card, does a DHCP Request, while also requesting, at least, options 66 and 67
  3. DHCP Server responds with an IP, and the two options
  4. The PXE Stack then tried to contact the server provided in option 66, to retrieve the file specified in option 67, which in this case is iPXE (for non-iPXE clients), over TFTP
  5. iPXE then unloads the native PXE stack (to a degree), and takes over, issuing it's own DHCP Request, again requesting, among other options, 66 and 67
  6. The DHCP Server responds with (typically) the same IP address, but now detects that the client is iPXE, and passes a different option 67.
  7. iPXE then boots to the URL passed in option 67 this time (via HTTP), and that script then directs it what to do.

Files

dhcpd.conf

  • This information can be placed globally, specific subnets, or individual filename/next-server can be placed on host definitions
  • Relevant sections of my ISC DHCP config:
1 if exists user-class {
2     if option user-class = "iPXE" {
3         filename "http://boot.example.com/preboot.php";
4     }
5 } else {
6         filename "ipxe.pxe";
7 }
8 next-server    10.0.3.2;
  • This if statement breaks the "infinite loop"
  • The filename listed on line 3 is for when iPXE does the DHCP request
  • The filename listed on line 6 is for when a non-iPXE client does a DHCP request, to get it to use iPXE
  • The next-server on line 8 is the TFTP server that the non-iPXE clients will grab the above file from
  • You can also use some iPXE specific DHCP options to gain more control over iPXE

ipxe.pxe

  • This file is built from the iPXE source, running "make bin/ipxe.pxe", then copying the bin/ipxe.pxe file to your TFTP server's root directory
  • More information on the extensions can be found here on the Etherboot website
  • As stated in the website linked above, pxe unloads the stack, which then means ipxe relies on it's own drivers for the network card. If you to keep the driver loaded, you may want to consider using .kpxe or .kkpxe

01_boot_vhost.conf

  • This is placed in /etc/apache2/vhosts.d, to configure the boot.example.com vhost
  • This configures your virtual host (boot.example.com from above), to point to where you want to store your files. I choose to keep mine in the TFTP root, under the "netboot" folder, so everything related to the netboot is contained in that folder.
1 <VirtualHost boot.example.com:80>
2         ServerName boot.example.com
3         DocumentRoot    "/tftp/netboot/"
4         <Directory "/tftp/netboot">
5             Options Indexes FollowSymLinks
6             AllowOverride All
7             Require all granted
8         </Directory>
9 </VirtualHost>

preboot.php

  • This is the file that the iPXE client first grabs to see what it needs to do
1 <?php 
2         echo "#!ipxe\n";
3         echo "chain http://".$_SERVER['SERVER_NAME']."/boot.php?MAC=\${netX/mac}";
4 ?>
  • So this is an extremely simple file, as you can tell. It's purpose is to pass the MAC address of the booting network card off to the boot.php script. So this references the same server name (pulls it in from the PHP variables), and then uses the ${netX/mac} iPXE variable to pull the MAC of the active network card, via GET (Variables in the URL).

boot.php

  • This is the big guy, so I'm going to do some inline comments to explain it
 1 <?php
 2 	// First, check the MAC, if not present, exit; no use continuing as we can assume the netboot isn't correct
 3  	if (isset($_GET['MAC'])) {
 4 		$mac = $_GET['MAC'];
 5 	} else {
 6 		exit();
 7 	}
 8 
 9 	header ( "Content-type: text/plain" );
10 	
11 	// Figure out which protocol we're using
12 	if (isset($_SERVER['HTTPS'])) {
13 		$serverproto = "https://";
14 	} else {
15 		$serverproto = "http://";
16 	}
17 	
18 	// If we're running a non-standard port, include it. Otherwise, assume defaults.
19 	if ( ($_SERVER['SERVER_PORT'] != "80") && ($_SERVER['SERVER_PORT'] != "443")) {
20 		$servername = "$_SERVER[SERVER_NAME]:$_SERVER[SERVER_PORT]";
21 	} else {
22 		$servername = $_SERVER['SERVER_NAME'];
23  	}

--

  • The above has some inline comments that should explain most of it. In my case, if there is no MAC, just quit, as it didn't run through preboot.php first, therefore is not correct
  • The rest of it is used to make the script portable. This allows you to use http/https, and also run on non-standard ports, and it can change at runtime. Perfect example of this is internal versus external booting, in reference to say your home network

--

24 switch ($mac) {
25 		case "00:0c:29:xx:xx:xx":
26 		case "00:e0:b8:xx:xx:xx":
27 			// Testing VM
28 			ipxemenu();
29 			break;
30 		case "bc:ae:c5:xx:xx:xx":
31 			echo "#!ipxe\n";
32 			// Atom - Living Room Frontend
33 			echo "kernel $serverproto$servername/bzImage-Atom ip=dhcp root=/dev/nfs nfsroot=10.0.3.2:/nfsroot/hdfe,nfsvers=3,tcp,hard,intr,nolock vga=0x317 quiet\n"; 
34 			// splash=silent,theme:MythTV quiet console=tty1\n";
35 			// echo "initrd $serverproto$servername/fe1-initrd.img\n";
36 			break;
37 		case "00:10:18:xx:xx:xx":
38 			echo "#!ipxe\n";
39 			// Athlon 64 HDFE - Bedroom Frontend
40 			echo "kernel $serverproto$servername/bzImage-A64 ip=dhcp root=/dev/nfs nfsroot=10.0.3.2:/nfsroot/hdfe1,nfsvers=3,tcp,hard,intr,nolock vga=0x317\n";
41 			//splash=silent,theme:MythTV quiet console=tty1\n";
42 			//echo "initrd $serverproto$servername/fe1-initrd.img\n";
43 			break;
44 		default:
45 			ipxemenu();
46 			break;
47 	}

--

  • This is where the fun begins!
  • First, it searches to see if there are special per-MAC instructions to do. In the above example, the first two MACs run the ipxemenu() function, that I was using for testing, from when I was switching away from a vesamenu
  • The next two are two different MythTV Frontends I have in our apartment. These netboot to a NFSRoot, so you can see how to have iPXE directly load Linux over HTTP
  • Then if all else fails (usual), run the ipxemenu() function to present the iPXE menu. (Here I use to have stuff to chain to pxelinux.0, and load a vesamenu up, but I recently switched to iPXE's menu. For pxelinux integration, see my work menu)

--

 48 function ipxemenu() {
 49 		echo "#!ipxe\n";
 50 		echo ":menustart\n";
 51 		echo "menu iPXE Boot Menu\n";
 52 		echo "item --gap --	Diagnostics\n";
 53 		echo "item memtest	Memtest86+\n";
 54 		echo "item dft		Drive Fitness Test\n";
 55 		echo "item spinrite	Spinrite\n";
 56 		echo "item hdt		Hardware Detection Tool\n";
 57 		echo "item b57		b57 Diags\n";
 58 		echo "item qwipe	Quick Wipe\n";
 59 		echo "item --gap --	Internet Boots\n";
 60 		echo "item ipxedemo	iPXE Demo Boot\n";
 61 		echo "item ritboot	rit.edu clone of boot.kernel.org\n";
 62 		echo "item --gap --	Live Systems\n";
 63 		echo "item gparted	GParted\n";
 64 		echo "item smartos	SmartOS\n";
 65 		echo "item --gap --	WinPE\n";
 66 		echo "item peboot	Windows PEBoot via SANBoot\n";
 67 		echo "item wimboot	Windows PEBoot via WIMBoot\n";
 68 		echo "item --gap --	Installers\n";
 69 		echo "item esxi5	ESXi 5.1 Installer\n";
 70 		echo "item esxi5sl	ESXi 5.0 Installer via PXELinux\n";
 71 		echo "item freedos	FreeDOS 1.1\n";
 72 		echo "item vesamenu	Vesamenu\n";
 73 		echo "item win7pro64	Windows 7 Professional AMD64 Install\n";
 74 		echo "item --gap -- 	Other PXE Systems\n";
 75 		echo "item wds		Windows Deployment Services\n";
 76 		echo "item shell 	iPXE Shell\n";
 77 		echo "item sanhook	Sanhook test on FreeNAS\n";
 78 		echo "choose os && goto \${os}\n";
 79 		echo ":shell\n";
 80 		echo "shell\n";
 81 		win7pro64();
 82 		esxi5sl();
 83 		sanhook();
 84 		qwipe();
 85 		b57();
 86 		freedos();
 87 		smartos();
 88 		gparted();
 89 		wds();
 90 		hdt();
 91 		memtest();
 92 		ipxedemo();
 93 		ritboot();
 94 		dft();
 95 		spinrite();
 96 		sanbootpe();
 97 		wimbootpe();
 98 		vesamenu();
 99 		esxi5();
100 		echo ":end\n";
101 	}

--

  • This is the function that creates the main iPXE menu
  • You can check the full syntax from the link above, but it's fairly simple
  • This creates the menu that is seen when the client is booted. It looks something like this:

Ipxemenu1.png Ipxemenu2.png

  • Once the menu is created, it calls the different functions to include the goto markers, otherwise the menu will fail is that option is selected. I choose to do it this way, with the intention of eventually adding options where the menu can be dynamic depending on the machine
  • I also added a :end goto marker, at the end, as I found pxelinux needed this to boot, otherwise it continued on through this menu (i.e. booting the option that was placed after it)
  • Most of the below, you can find really good documentation on the iPXE website. However, I will note anything special as needed

--

102 	function win7pro64() {
103 		global $servername,$serverproto;
104 		echo ":win7pro64\n";
105 		echo "kernel $serverproto$servername/wimboot\n";
106 		echo "initrd $serverproto$servername/win7instadk/bootmgr bootmgr\n";
107 		echo "initrd $serverproto$servername/win7instadk/Fonts/segmono_boot.ttf segmono_boot.ttf\n";
108 		echo "initrd $serverproto$servername/win7instadk/Fonts/segoe_slboot.ttf segoe_slboot.ttf\n";
109 		echo "initrd $serverproto$servername/win7instadk/Fonts/wgl4_boot.ttf wgl4_boot.ttf\n";
110 		echo "initrd $serverproto$servername/win7instadk/Fonts/kor_boot.ttf kor_boot.ttf\n";
111 		echo "initrd $serverproto$servername/win7instadk/Fonts/wgl4_boot.ttf wgl4_boot.ttf\n";
112 		echo "initrd $serverproto$servername/win7instadk/BCD BCD\n";
113 		echo "initrd $serverproto$servername/win7instadk/boot.sdi boot.sdi\n";
114 		echo "initrd $serverproto$servername/win7instadk/boot.wim boot.wim\n";
115 		echo "boot\n";	
116 	}

--

  • This uses wimboot to boot the WIM file for Windows PE Environments. This particular one was created with the Windows ADK (WinPE 4.0), and has a script to mount a SMB Share that contains the Windows 7 installer. This will be detailed in another entry, and linked here at that time

--

117 	function sanhook() {
118 		echo ":sanhook\n";
119 		echo "sanhook iscsi:10.0.3.101::::iqn.2011-03.example.org.istgt:test\n";
120 		echo "sleep 5\n";
121 		echo "goto menustart\n";
122 	}
123 	function esxi5sl() {
124 		global $servername,$serverproto;
125 		echo ":esxi5sl\n";
126 		echo "set 210:string $serverproto$servername/esxi5/\n";
127 		echo "set 209:string isolinux.cfg\n";
128 		echo "chain $serverproto$servername/pxelinux.0\n";
129 		echo "goto end\n";
130 	}

--

  • This boots the ESXi installer via Syslinux (pxelinux/mboot). I found this way currently works.
  • This will be documented as other files, below.

--

131 	function qwipe() {
132 		global $servername,$serverproto;
133 		echo ":qwipe\n";
134 		echo "sanboot --drive 0x00 --no-describe $serverproto$servername/wipe2.img\n";
135 		echo "goto menustart\n";
136 	}
137 	function b57() {
138 		global $servername,$serverproto;
139 		echo ":b57\n";
140 		echo "sanboot --drive 0xa0 --no-describe $serverproto$servername/B57udiag-15.23.iso\n";
141 		echo "goto menustart\n";
142 	}
143 	function freedos() {
144 		global $servername,$serverproto;
145 		echo ":freedos\n";
146 		echo "sanboot --drive 0xa0 --no-describe $serverproto$servername/fd11src.iso\n";
147 		echo "goto menustart\n";
148 	}
149 	function smartos() {
150 		echo ":smartos\n";
151 		echo "kernel smartos/platform/i86pc/kernel/amd64/unix -B console=text,standalone=true,noimport=true,root_shadow='\$5\$2HOHRnK3\$NvLlm.1KQBbB0WjoP7xcIwGnllhzp2HnT.mDO7DpxYA'\n";
152 		echo "module smartos/platform/i86pc/amd64/boot_archive\n";
153 		echo "boot\n";
154 		echo "sleep 10\n";
155 	}
156 	function wds() {
157 		echo ":wds\n";
158 		echo "set wdsserver:ipv4 10.0.3.99\n";
159 		echo "set net0/next-server \${wdsserver}\n";
160 		echo "chain tftp://\${wdsserver}/boot%5Cx86%5cwdsnbp.com\n";
161 		echo "sleep 10\n";
162 		echo "goto  menustart\n";
163 	}

--

  • This one a lot of people may be interested in. This will chain to a Windows Deployment Services.
  • Things to note here, the wdsnbp.com looks at the next-server in the dhcp packets for where to talk to, well, this may not actually be your windows server, it is not in my case. This sets the next-server to the WDS Server, then chains to it, so that it boots properly.
  • Also note the hex in the tftp URL. This is needed, as it's a Windows TFTP Server, so it's actually in the path of boot\x86\wdsnbp.com
  • TODO? Use iPXE arch detection to boot 64bit?

--

164 	function gparted() {
165 		global $servername,$serverproto;
166 		echo ":gparted\n";
167 		echo "kernel $serverproto$servername/gparted/live/vmlinuz boot=live config union=aufs noswap noprompt fetch=$serverproto$servername/gparted/live/filesystem.squashfs\n"; 
168 		echo "initrd $serverproto$servername/gparted/live/initrd.img\n";
169 		echo "boot\n";
170 		echo "sleep 10\n";
171 		echo "goto menustart\n";
172 	}
173 	function hdt() {
174 		global $servername,$serverproto;
175 		echo ":hdt\n";
176 		echo "sanboot --no-describe --drive 0x00 $serverproto$servername/hdt-0.5.0.img\n";
177 		echo "goto menustart\n";
178 	}
179 	function spinrite() {
180 		global $servername,$serverproto;
181 		echo ":spinrite\n";
182 		echo "sanboot --drive 0xa1 --no-describe $serverproto$servername/spinrite.iso\n";
183 		echo "goto menustart\n";
184 	}
185 	function ipxedemo() {
186 		echo ":ipxedemo\n";
187 		echo "chain http://boot.ipxe.org/demo/boot.php\n";
188 	}
189 	function ritboot() {
190 		echo ":ritboot\n";
191 		echo "set 209:string pxelinux.cfg/default\n";
192 		echo "set 210:string http://boot.rit.edu/bko/\n";
193 		echo "chain http://boot.rit.edu/bko/pxelinux.0\n";
194 		echo "goto :menustart\n";
195 	}
196 	function dft() {
197 		global $servername,$serverproto;
198 		echo ":dft\n";
199 		echo "sanboot --drive 0x00 $serverproto$servername/dft32_v416_b00_install.IMG\n";
200 	}
201 	
202 	function memtest() {
203 		global $servername,$serverproto;
204 		echo ":memtest\n";
205 		echo "chain $serverproto$servername/memtest.0\n";
206 	}

--

--

207 	function sanbootpe() {
208 		global $servername,$serverproto;
209 		echo ":peboot\n";
210 		echo "set keep-san 1\n";
211 		echo "sanboot  --drive 0x81 $serverproto$servername/PEBoot-07_08_11.iso\n";
212 	}
213 	function wimbootpe() {
214 		global $servername,$serverproto;
215 		echo ":wimboot\n";
216 		echo "kernel $serverproto$servername/wimboot\n";
217 		echo "initrd $serverproto$servername/winpe/Boot/bootmgr.exe bootmgr.exe\n";
218 		echo "initrd $serverproto$servername/winpe/Boot/BCD BCD\n";
219 		echo "initrd $serverproto$servername/winpe/Boot/boot.sdi boot.sdi\n";
220 		echo "initrd $serverproto$servername/winpe/Boot/boot.wim boot.wim\n";
221 		echo "boot\n";	
222 	}

--

  • wimboot is a much better way of booting WinPE via iPXE. It's extremely quick, and saves memory!

--

223 	function vesamenu() {
224 		global $servername,$serverproto;
225 		echo ":vesamenu\n";
226 		echo "set 210:string $serverproto$servername/\n";
227 		echo "set 209:string mainmenu.gpxe\n";
228 		echo "chain \${210:string}pxelinux.0\n";
229 	}
230 	function esxi5() {
231 		global $servername,$serverproto;
232 		echo ":esxi5\n";
233 		echo "chain $serverproto$servername/esxi51/esxi51.ipxe\n";
234 		echo "goto end\n";
235 	}
236 ?>

--

  • Chains to an ipxe config in the esxi51 directory, to load the ESXi installer.

--

isolinux.cfg

  • I made a quick and simple file that loads the installer immediately, instead of hitting the default boot menu
  • In addition to the above, you also need to make some other modifications (that link also contains the other part of the instructions to get it all setup -- basic instructions to get the installer to PXE boot)
1 DEFAULT install
2 LABEL install
3   KERNEL mboot.c32
4   APPEND -c boot.cfg

esxi51.ipxe

 1 #!ipxe
 2 imgfree
 3 kernel tboot runweasel
 4 module b.b00
 5 module k.b00
 6 module chardevs
 7 module a.b00
 8 module user.b00
 9 module s.v00
10 module ata_pata.v00
11 module ata_pata.v01
12 module ata_pata.v02
13 module ata_pata.v03
14 module ata_pata.v04
15 module ata_pata.v05
16 module ata_pata.v06
17 module ata_pata.v07
18 module block_cc.v00
19 module ehci_ehc.v00
20 module weaselin.t00
21 module esx_dvfi.v00
22 module xlibs.v00
23 module ima_qla4.v00
24 module ipmi_ipm.v00
25 module ipmi_ipm.v01
26 module ipmi_ipm.v02
27 module misc_cni.v00
28 module misc_dri.v00
29 module net_be2n.v00
30 module net_bnx2.v00
31 module net_bnx2.v01
32 module net_cnic.v00
33 module net_e100.v00
34 module net_e100.v01
35 module net_enic.v00
36 module net_forc.v00
37 module net_igb.v00
38 module net_ixgb.v00
39 module net_nx_n.v00
40 module net_r816.v00
41 module net_r816.v01
42 module net_s2io.v00
43 module net_sky2.v00
44 module net_tg3.v00
45 module net_vmxn.v00
46 module ohci_usb.v00
47 module sata_ahc.v00
48 module sata_ata.v00
49 module sata_sat.v00
50 module sata_sat.v01
51 module sata_sat.v02
52 module sata_sat.v03
53 module sata_sat.v04
54 module scsi_aac.v00
55 module scsi_adp.v00
56 module scsi_aic.v00 
57 module scsi_bnx.v00
58 module scsi_fni.v00
59 module scsi_hps.v00
60 module scsi_ips.v00
61 module scsi_lpf.v00
62 module scsi_meg.v00
63 module scsi_meg.v01
64 module scsi_meg.v02
65 module scsi_mpt.v00
66 module scsi_mpt.v01
67 module scsi_mpt.v02
68 module scsi_qla.v00
69 module scsi_qla.v01
70 module scsi_rst.v00
71 module uhci_usb.v00
72 module tools.t00
73 module xorg.v00
74 module imgdb.tgz
75 module imgpayld.tgz
76 boot

Resources Used

Fin

These files may end up on my github at some point. If/when they do, I'll note it here.

iPXE is EXTREMELY powerful, especially with the embedded scripting. There is so much that can be done, it's hard to go into details, just because what you can do is so vast! A perfect example of this is Robin's iPXE menu linked to above. He does some awesome stuff!

I'm always looking to expand the netboot. Be adding diag utils, playing around with new options, or just plain tweaking it. In the above boot.php, a few of the things I want to tweak, is instead of referencing the name boot.example.com all the time, have it pull the PHP server variable. This way if you have a different hostname internal/external, as long as you set it once (say in a script, or via DHCP), it will always reference that server.

One thing to note with iPXE, is you can also embed the scripts into it. In my work netboot (coming soon!), I embed pxelinux.0 and a small script. This will be explained more in that post, but one thing to note, is you can embed a script that does DHCP, then chains to your server at home. As long as you have external access to that box, and flash the resulting iPXE to a USB drive, or CD, you can boot any machine anywhere in the world.

For contacting myself (for complaints/improvements/suggestions), or others who use iPXE, there is the IRC channel, irc.freenode.net, #ipxe. I'm pretty much always in there as Sedorox. The developers also hang out there, and other users too, who do all sorts of fun network booting things. Come visit!

Hopefully this helps someone better understand iPXE and netbooting, or at the very least, serves as an example on what you can do.