машина тестовая для перевода PPPoE сервера шейпинга с ipfw+dummynet на ng_bpf+ng_car

# uname -a
FreeBSD nas3.inet72.ru 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008     root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
ядро было самосборное.

# kldstat
Id Refs Address    Size     Name
1   17 0xc0400000 906518   kernel
2    1 0xc0d07000 58e0     if_vlan.ko
3    1 0xc0d0d000 30718    if_em.ko
4    1 0xc0d3e000 3df8     ng_ether.ko
5    5 0xc0d42000 d3a0     netgraph.ko
6    1 0xc0d50000 8ab4     if_lagg.ko
7    1 0xc0d59000 2a90     ichwd.ko
8    1 0xc0d5c000 295c     coretemp.ko
9    1 0xc0d5f000 6a32c    acpi.ko
10    1 0xc31fd000 4000     ng_socket.ko
11    1 0xc326d000 4000     ng_mppc.ko
12    1 0xc3271000 2000     rc4.ko
13    1 0xc327a000 5000     ng_pppoe.ko

if_em не загрузился по причине присутствия оного в GENERIC ядре

на всякий случай dmesg:

предидущая загрузка:
ad0: 76318MB <Seagate ST380011A 8.01> at ata0-master UDMA100
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/ad0s1a
WARNING: attempt to net_add_domain(netgraph) after domainfinalize()
em0: link state changed to UP
vlan100: link state changed to UP
vlan7: link state changed to UP

Fatal double fault:
eip = 0xc0d160e0
esp = 0xd546df98
ebp = 0xd546e250
cpuid = 0; apic id = 00
panic: double fault
cpuid = 0
Uptime: 10m0s
Physical memory: 498 MB
Dumping 49 MB: 34 18 2Copyright (c) 1992-2008 The FreeBSD Project.
watchdog прервал дамп... сейчас отключу wd его и попробую сного
загрука:
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008
    root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
module_register: module pci/em already exists!
Module pci/em failed to register: 17
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Pentium(R) D  CPU 2.66GHz (2676.15-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf47  Stepping = 7
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x651d<SSE3,RSVD2,MON,DS_CPL,TM2,CNXT-ID,CX16,xTPR>
  AMD Features=0x20000000<LM>
  AMD Features2=0x1<LAHF>
  Cores per package: 2
real memory  = 536346624 (511 MB)
avail memory = 510898176 (487 MB)
ACPI APIC Table: <A M I  OEMAPIC >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
ichwd module loaded
ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
hptrr: HPT RocketRAID controller driver v1.1 (Feb 24 2008 19:59:27)
acpi0: <A M I OEMXSDT> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, 1ff00000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
cpu0: <ACPI CPU> on acpi0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
p4tcc1: <CPU Frequency Thermal Control> on cpu1
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci3: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
pci2: <ACPI PCI bus> on pcib2
mskc0: <Marvell Yukon 88E8053 Gigabit Ethernet> port 0xd800-0xd8ff mem 0xefffc000-0xefffffff irq 16 at device 0.0 on pci2
msk0: <Marvell Technology Group Ltd. Yukon EC Id 0xb6 Rev 0x02> on mskc0
msk0: Ethernet address: 00:17:31:3e:d5:48
miibus0: <MII bus> on msk0
e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 0 on miibus0
e1000phy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
mskc0: [FILTER]
pcib3: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib3
em0: <Intel(R) PRO/1000 Network Connection Version - 6.7.3> port 0xc800-0xc83f mem 0xefec0000-0xefedffff,0xefea0000-0xefebffff irq 21 at device 0.0 on pci1
em0: Ethernet address: 00:07:e9:09:d8:08
em0: [FILTER]
vgapci0: <VGA-compatible display> mem 0xe8000000-0xebffffff irq 23 at device 2.0 on pci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH7 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci0
ata1: [ITHREAD]
atapci1: <Intel ICH7 SATA300 controller> port 0xb800-0xb807,0xb400-0xb403,0xb000-0xb007,0xa800-0xa803,0xa400-0xa40f mem 0xe7fffc00-0xe7ffffff irq 23 at device 31.2 on pci0
atapci1: [ITHREAD]
ata2: <ATA channel 0> on atapci1
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci1
ata3: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_button0: <Power Button> on acpi0
fdc0: <floppy drive controller (FDE)> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FILTER]
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio0: [FILTER]
pmtimer0 on isa0
ichwd0: <Intel ICH7 watchdog timer> on isa0
ichwd0: Intel ICH7 watchdog timer (ICH7 or equivalent)
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff pnpid ORM0000 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
hptrr: no controller detected.
ad0: 76318MB <Seagate ST380011A 8.01> at ata0-master UDMA100
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/ad0s1a
WARNING: / was not properly dismounted
em0: link state changed to UP
vlan100: link state changed to UP
vlan7: link state changed to UP
WARNING: attempt to net_add_domain(netgraph) after domainfinalize()

# ngctl list
There are 22 total nodes:
  Name: ngctl964        Type: socket          ID: 0000001a   Num hooks: 0
  Name: <unnamed>       Type: car             ID: 00000019   Num hooks: 2
  Name: <unnamed>       Type: car             ID: 00000018   Num hooks: 2
  Name: <unnamed>       Type: car             ID: 00000017   Num hooks: 2
  Name: <unnamed>       Type: car             ID: 00000016   Num hooks: 2
  Name: mpd665-B_pppoe-1-lim Type: bpf             ID: 00000015   Num hooks: 18
  Name: mpd665-B_pppoe-1-mss Type: tcpmss          ID: 00000014   Num hooks: 2
  Name: mpd665-nfso     Type: ksocket         ID: 00000013   Num hooks: 1
  Name: mpd665-nf       Type: netflow         ID: 00000012   Num hooks: 5
  Name: mpd665-stats    Type: socket          ID: 00000010   Num hooks: 0
  Name: mpd665-B_pppoe-1 Type: ppp             ID: 0000000f   Num hooks: 3
  Name: ng0             Type: iface           ID: 0000000e   Num hooks: 1
  Name: mpd665-L_pppoe-1-lt Type: tee             ID: 0000000d   Num hooks: 2
  Name: <unnamed>       Type: pppoe           ID: 0000000b   Num hooks: 3
  Name: <unnamed>       Type: socket          ID: 0000000a   Num hooks: 1
  Name: mpd665-eso      Type: socket          ID: 00000009   Num hooks: 0
  Name: mpd665-cso      Type: socket          ID: 00000008   Num hooks: 0
  Name: mpd665-lso      Type: socket          ID: 00000007   Num hooks: 1
  Name: vlan100         Type: ether           ID: 00000004   Num hooks: 1
  Name: vlan7           Type: ether           ID: 00000003   Num hooks: 0
  Name: em0             Type: ether           ID: 00000002   Num hooks: 0
  Name: msk0            Type: ether           ID: 00000001   Num hooks: 0

ngctl show "[15]:"
  Name: mpd665-B_pppoe-1-lim Type: bpf             ID: 00000015   Num hooks: 18
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  1-1-ni          mpd665-B_pppoe-1-lim bpf          00000015        1-1-n
  1-1-n           mpd665-B_pppoe-1-lim bpf          00000015        1-1-ni
  1-1-mi          <unnamed>       car          00000019        lower
  1-1-m           <unnamed>       car          00000019        upper
  1-0-ni          mpd665-B_pppoe-1-lim bpf          00000015        1-0-n
  1-0-n           mpd665-B_pppoe-1-lim bpf          00000015        1-0-ni
  1-0-mi          <unnamed>       car          00000018        lower
  1-0-m           <unnamed>       car          00000018        upper
  0-1-ni          mpd665-B_pppoe-1-lim bpf          00000015        0-1-n
  0-1-n           mpd665-B_pppoe-1-lim bpf          00000015        0-1-ni
  0-1-mi          <unnamed>       car          00000017        upper
  0-1-m           <unnamed>       car          00000017        lower
  0-0-ni          mpd665-B_pppoe-1-lim bpf          00000015        0-0-n
  0-0-n           mpd665-B_pppoe-1-lim bpf          00000015        0-0-ni
  0-0-mi          <unnamed>       car          00000016        upper
  0-0-m           <unnamed>       car          00000016        lower
  iface           ng0             iface        0000000e        inet
  ppp             mpd665-B_pppoe-1-mss tcpmss       00000014        out

в веб-интерфейсе mpd:
Traffic filters:
    1#1    : 'match src host 90.20.37.96 and dst net 90.20.36.0/22'
    2#2    : 'match dst host 90.20.37.96 and src net 90.20.36.0/22'
    3#1    : 'match src host 90.20.37.96 and dst net 0.0.0.0/0'
    4#2    : 'match dst host 90.20.37.96 and src net 0.0.0.0/0'
Traffic limits:
    in#1    : 'flt1 rate-limit 10485760 pass'
    in#3    : 'flt3 rate-limit 5242880 pass'
    in#5    : 'all deny'
    out#2    : 'flt2 rate-limit 10485760 pass'
    out#4    : 'flt4 rate-limit 5242880 pass'
    out#6    : 'all deny'

проблема остаётся, если будет:
in#1 : 'all rate-limit 10485760 pass'
out#2: 'all rate-limit 10485760 pass'

msk0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=19a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
        ether 00:17:31:3e:d5:48
        media: Ethernet autoselect (none)
        status: no carrier
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 00:07:e9:09:d8:08
        media: Ethernet autoselect (1000baseTX <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet6 ::1 prefixlen 128
        inet 127.0.0.1 netmask 0xff000000
vlan7: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=3<RXCSUM,TXCSUM>
        ether 00:07:e9:09:d8:08
        inet 90.20.39.241 netmask 0xfffffe00 broadcast 91.203.39.255
        inet 90.20.37.6 netmask 0xffffff00 broadcast 91.203.37.255
        media: Ethernet autoselect (1000baseTX <full-duplex>)
        status: active
        vlan: 7 parent interface: em0
vlan100: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=3<RXCSUM,TXCSUM>
        ether 00:07:e9:09:d8:08
        media: Ethernet autoselect (1000baseTX <full-duplex>)
        status: active
        vlan: 100 parent interface: em0
ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1480
        inet 2.2.2.2 --> 90.20.37.96 netmask 0xffffffff

arp -n: (в mpd включен proxy-arp)
? (90.20.37.96) at 00:07:e9:09:d8:08 on vlan7 permanent published (proxy only) [vlan]

машину ребутит watchdog (машина на другом конце города)
только что вернул на место GENERIC ядро, вот коредамп мне сделал корку, буду разбираться,

но всё-таки
есть идеи?