在 FreeBSD 建立樣版功能的 jail 環境

0. 準備步驟 :
0.0. 準備 Host 的網路環境
0.0.0. 綁 IP 到 Host : 編輯 /etc/rc.conf 加上 IP alias 設定
ifconfig_em0_alias1="192.168.66.66 netmask 255.255.255.255"

0.0.1. 設定 natd (如果是只有一片網卡的機器就可以不用, 此步驟適用機器有跨在兩個網路(一內一外)上)
Edit /etc/rc.conf

gateway_enable="YES"

firewall_enable="YES"
firewall_type="/etc/my.firewall"
firewall_script="/etc/my.firewall"
firewall_logging="YES"

natd_enable="YES"
natd_interface="re0"
natd_flags="-l -f /etc/natd.conf"


Edit /etc/my.firewall (請依照實際需求更改)

#!/bin/sh -

fwcmd="/sbin/ipfw"                                                                                               

INTIF="re0"

LANIF0="em0"

${fwcmd} -f flush                                                                                              
#For NATD                                                                                                      
${fwcmd} add 00030 divert natd all from any to any via ${INTIF}
...
${fwcmd} add 65000 pass all from any to any

Edit /etc/natd.conf

use_sockets yes
same_ports yes
dynamic yes

Edit /etc/services

natd            8668/divert # Network Address Translation

接著 Host 重新啟動.

0.1. 在 /usr/src 編譯整個環境.
# cd /usr/src; make buildworld

1. 設定樣板存放位置且安裝樣板

# zfs create storage/jail
# zfs set mountpoint=/storage/jail storage/jail
# zfs set dedup=on storage/jail
# zfs set quota=300G storage/jail

# zfs create storage/jails
# zfs set mountpoint=/storage/jails storage/jails
# zfs set dedup=on storage/jails
# zfs set quota=300G storage/jails

# mkdir -p /storage/jail/mroot

# cd /usr/src
# make installworld DESTDIR=/storage/jail/mroot

2. 複製 src 及 ports 
# cd /storage/jail/mroot
# mkdir usr/ports
# portsnap -p /storage/jail/mroot/usr/ports fetch extract
# cpdup /usr/src /storage/jail/mroot/usr/src

3. 建立樣板環境
# mkdir /storage/jail/skel /storage/jail/skel/home /storage/jail/skel/usr-X11R6 /storage/jail/skel/distfiles /storage/jail/skel/packages
# mv etc /storage/jail/skel
# mv usr/local /storage/jail/skel/usr-local
# mv tmp /storage/jail/skel
# mv var /storage/jail/skel
# mv root /storage/jail/skel

4. 產生設定檔
# mergemaster -t /storage/jail/skel/var/tmp/temproot -D /storage/jail/skel -i
# cd /storage/jail/skel
# rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev

5. 建立個別環境可寫入的目錄
#cd /storage/jail/mroot
#mkdir s
#ln -s s/etc etc
#ln -s s/home home
#ln -s s/root root
#ln -s ../s/usr-local usr/local
#ln -s ../s/usr-X11R6 usr/X11R6
#ln -s ../../s/distfiles usr/ports/distfiles
#ln -s ../../s/packages usr/ports/packages
#ln -s s/tmp tmp
#ln -s s/var var

6. 建立 jailed O.S. 的 make.conf
# Add /storage/jail/skel/etc/make.conf
# WRKDIRPREFIX?=  /s/portbuild

以下是每個 jailed O.S. 都要做一次

7. 設定 Host 每次開機為 jailed O.S. 掛上目錄
[FreeBSD 9 labdata2] (Why: 開機時因 ZFS 來不及掛載而無法在 /etc/fstab 處理)
Edit /etc/rc.local
### For jail
/sbin/mount -t nullfs -r /storage/jail/mroot /storage/jail/labdata2
/sbin/mount -t nullfs -rw /storage/jails/labdata2 /storage/jail/labdata2/s

[FreeBSD 8 localdata2]
Edit /etc/fstab

#/storage/jail/mroot   /storage/jail/localdata2     nullfs  ro  0   0
#/storage/jails/localdata2     /storage/jail/localdata2/s   nullfs  rw  0   0


8. 編輯 Host 的 /etc/rc.conf
###20121004
jail_enable="YES"
jail_set_hostname_allow="NO"
### 以下是清單, 有幾台 jailed O.S. 就登記幾台
jail_list="localdata2"
### 以下是每一台 jailed O.S. 的設定記得每一台都要有一組設定.(前四行是必要的)
jail_localdata2_hostname="localdata2.DOMAIN_NAME"
jail_localdata2_ip="192.168.66.66"
jail_localdata2_interface="em0"
jail_localdata2_rootdir="/storage/jail/localdata2"
jail_localdata2_devfs_enable="YES"
### 20121113 解決 netstat -rn 查詢問題
#jail_localdata2_devfs_ruleset="devfsrules_jail"
jail_localdata2_devfs_ruleset="devfsrules_jail2"
jail_localdata2_exec_start="/bin/sh /etc/rc"
jail_localdata2_exec_stop="/bin/sh /etc/rc.shutdown"
jail_localdata2_fdescfs_enable="YES"
jail_localdata2_procfs_enable="YES"

9. 建立系統目錄
mkdir /storage/jail/localdata2
mkdir /storage/jails/localdata2
cpdup /storage/jail/skel /storage/jails/localdata2
mkdir /storage/jail/localdata2/s

10. [Optional] 在 Host 編輯 /etc/sysctl.conf 開放 jailed O.S. 使用 ping.
Edit /etc/sysctl.conf
###20121113 For jail to fix ping icmp socket: Operation not permitted
security.jail.allow_raw_sockets=1

11. [Optional] 在 Host 編輯 /etc/default/devfs.rules 開放 jailed O.S. 使用 pts pty mem bpf
11.1. Edit /etc/default/devfs.rules
[devfsrules_unhide_login=3]
...
add path pts unhide
add path 'pts/*' unhide
add path pty unhide
add path 'pty/*' unhide

11.2. Edit /etc/devfs.rules
###
### http://forums.freebsd.org/archive/index.php/t-24581.html
### http://forums.freebsd.org/archive/index.php/t-5693.html
###
###

[devfsrules_unhide_mem=5]
add path mem unhide
add path kmem unhide

[devfsrules_unhide_bpf=6]
add path bpf unhide

# Devices usually found in a jail.
#
[devfsrules_jail2=7]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add include $devfsrules_unhide_mem
add include $devfsrules_unhide_bpf

12. 掛載目錄
[FreeBSD 9]

/sbin/mount -t nullfs -r /storage/jail/mroot /storage/jail/labdata2
/sbin/mount -t nullfs -rw /storage/jails/labdata2 /storage/jail/labdata2/s


[FreeBSD 8]
mount /storage/jail/localdata2
mount /storage/jail/localdata2/s

13. 啟動 jail
# /etc/rc.d/jail start
查詢啟動的 jailed O.S.
# jls
進入 jailed O.S.
# jexec #JID tcsh (或 jexec #JID /ur/bin/su -)

14. 編輯 jailed O.S. 的系統設定
Edit /etc/rc.conf
Edit /etc/resolv.conf
...


參考資料
1. FreeBSD 使用手冊  15.6 Jail 的應用
2. Forums.freebsd.org :  [Solved] jails with vnet in rc.conf
3. Forums.freebsd.org : jail nat route