To wire this driver into your kernel,
add the following line to your kernel configuration file:
device lmc
Alternatively, to load this module at boot time, add
if_lmc_load="YES"
to
/boot/loader.conf
see
loader.conf5.
To wire a line protocol into your kernel, add:
options NETGRAPHdevice sppp
It is not necessary to wire line protocols into your kernel,
they can be loaded later with
kldload(8).
The driver can send and receive raw IP packets even if neither
SPPP nor Netgraph are configured into the kernel.
Netgraph and SPPP can both be enabled; Netgraph will be used if the
rawdata
hook is connected.
DESCRIPTION
This is an open-source
UNIX
device driver for PCI-bus WAN interface cards.
It sends and receives packets in HDLC frames over synchronous circuits.
A generic PC plus
UNIX
plus some
LMC / SBE
cards makes an
open
router.
This driver works with
Fx ,
Nx ,
Ox ,
Bs x
and
Linux
OSs.
It has been tested on i386 (SMP 32-bit little-endian) and Sparc (64-bit big-endian)
architectures.
The
driver works with the following cards:
SBE wanADAPT-HSSI (LMC5200)
High Speed Serial Interface,
EIA612/613, 50-pin connector,
0 to 52 Mb/s, DTE only.
SBE wanADAPT-T3 (LMC5245)
T3: two 75-ohm BNC connectors,
C-Parity or M13 Framing,
44.736 Mb/s, up to 950 ft.
SBE wanADAPT-SSI (LMC1000)
Synchronous Serial Interface,
V.35, X.21, EIA449, EIA530(A), EIA232,
0 to 10 Mb/s, DTE or DCE.
SBE wanADAPT-T1E1 (LMC1200)
T1 or E1: RJ45 conn, 100 or 120 ohms,
T1-ESF-B8ZS, T1-SF-AMI, E1-(many)-HDB3,
1.544 Mb/s or 2.048 Mb/s, up to 6 Kft.
Cards contain a high-performance
PCI
interface, an
HDLC
function and
either integrated
modems
(T1, T3) or
modem
interfaces (HSSI and SSI).
PCI
The PCI interface is a DEC 21140A "Tulip" Fast Ethernet chip.
This chip has an efficient PCI implementation with scatter/gather DMA,
and can run at 100 Mb/s full duplex (twice as fast as needed here).
HDLC
The HDLC functions (ISO-3309: flags, bit-stuffing, CRC) are implemented
in a Field Programmable Gate Array (FPGA) which talks to the Ethernet
chip through a Media Independent Interface (MII).
The hardware in the FPGA translates between Ethernet packets and
HDLC frames on-the-fly; think it as a WAN PHY chip for Ethernet.
Modem
The modem chips are the main differences between cards.
HSSI cards use ECL10K chips to implement the EIA-612/613 interface.
T3 cards use a TranSwitch TXC-03401 framer chip.
SSI cards use Linear Technology LTC1343 modem interface chips.
T1 cards use a BrookTree/Conexant/Mindspeed Bt8370 framer
and line interface chip.
Line protocols exist above device drivers and below internet protocols.
They typically encapsulate packets in HDLC frames and deal with
higher-level issues like protocol multiplexing and security.
This driver is compatible with several line protocol packages:
Netgraph
Netgraph(4)
implements many basic packet-handling functions as kernel loadable modules.
They can be interconnected in a graph to implement many protocols.
Configuration is done from userland without rebuilding the kernel.
Packets are sent and received through this interface if the driver's
rawdata
hook is connected, otherwise the ifnet interface (SPPP and RawIP) is used.
ASCII configuration control messages are
not
currently supported.
SPPP
sppp(4)
implements Synchronous-PPP, Frame-Relay and Cisco-HDLC in the kernel.
RawIP
This null line protocol, built into the driver, sends and receives
raw IPv4 and IPv6 packets in HDLC frames (aka IP-in-HDLC) with
no extra bytes of overhead and no state at the end points.
EXAMPLES
ifconfig and lmcconfig
The program
lmcconfig(8)
manipulates interface parameters beyond the scope of
ifconfig(8).
In normal operation only a few arguments are needed:
-X
selects the external
SPPP
line protocol package.
-x
selects the built-in RawIP line protocol package.
-Z
selects PPP line protocol.
-z
selects Cisco-HDLC line protocol.
-F
selects Frame-Relay line protocol.
lmcconfig lmc0
displays interface configuration and status.
lmcconfig lmc0 -D
enables debugging output from the device driver only.
ifconfig lmc0 debug
enables debugging output from the device driver and from
the line protocol module above it.
Debugging messages that appear on the console are also
written to file
/var/log/messages
Caution
when things go very wrong, a torrent of debugging messages
can swamp the console and bring a machine to its knees.
If the driver is unloaded and then loaded, reconnect hooks by:
"ngctl connect lmc0: ng0: rawdata inet"
TESTING
Testing with Loopbacks
Testing with loopbacks requires only one card.
Packets can be looped back at many points: in the PCI chip,
in the modem chips, through a loopback plug, in the
local external equipment, or at the far end of a circuit.
All cards can be looped through the PCI chip.
Cards with internal modems can be looped through
the modem framer and the modem line interface.
Cards for external modems can be looped through
the driver/receiver chips.
See
lmcconfig(8)
for details.
Loopback plugs test everything on the card.
HSSI
Loopback plugs can be ordered from SBE (and others).
Transmit clock is normally supplied by the external modem.
When an HSSI card is operated with a loopback plug, the PCI bus
clock must be used as the transmit clock, typically 33 MHz.
When testing an HSSI card with a loopback plug,
configure it with
lmcconfig(8):
"lmcconfig lmc0 -a 2"
``-a 2
''
selects the PCI bus clock as the transmit clock.
T3
Connect the two BNC jacks with a short coax cable.
SSI
Loopback plugs can be ordered from SBE (only).
Transmit clock is normally supplied by the external modem.
When an SSI card is operated with a loopback plug,
the on-board clock synthesizer must be used.
When testing an SSI card with a loopback plug,
configure it with
lmcconfig(8):
"lmcconfig lmc0 -E -f 10000000"
-E
puts the card in DCE mode to source a transmit clock.
``-f 10000000
''
sets the internal clock source to 10 Mb/s.
T1/E1
A loopback plug is a modular plug with two wires
connecting pin 1 to pin 4 and pin 2 to pin 5.
One can also test by connecting to a local modem (HSSI and SSI)
or NI (T1 and T3) configured to loop back.
Cards can generate signals to loopback remote equipment
so that complete circuits can be tested; see
lmcconfig(8)
for details.
Testing with a Modem
Testing with a modem requires two cards of different types.
T3/HSSI
If you have a T3 modem with an HSSI interface
(made by Digital Link, Larscom, Kentrox etc.)
then use an HSSI card in one machine and a T3 card in the other machine.
The T3 coax cables must use the null modem configuration (see below).
T1/V.35
If you have a T1 (or E1) modem with a V.35, X.21 or EIA530 interface,
then use an SSI card in one machine and a T1 card in the other machine.
Use a T1 null modem cable (see below).
Testing with a Null Modem Cable
Testing with a null modem cable requires two cards of the same type.
HSSI
Three-meter HSSI null-modem cables can be ordered from SBE.
In a pinch, a 50-pin SCSI-II cable up to a few meters will
work as a straight HSSI cable (not a null modem cable).
Longer cables should be purpose-built HSSI cables because
the cable impedance is different.
Transmit clock is normally supplied by the external modem.
When an HSSI card is connected by a null modem cable, the PCI bus
clock can be used as the transmit clock, typically 33 MHz.
When testing an HSSI card with a null modem cable, configure it
with
lmcconfig(8):
"lmcconfig lmc0 -a 2
``-a 2
''
selects the PCI bus clock as the transmit clock.
T3
T3 null modem cables are just 75-ohm coax cables with BNC connectors.
TX OUT on one card should be connected to RX IN on the other card.
In a pinch, 50-ohm thin Ethernet cables
usually
work up to a few meters, but they will
not
work for longer runs [em] 75-ohm coax is
required
SSI
Three-meter SSI null modem cables can be ordered from SBE.
An SSI null modem cable reports a cable type of V.36/EIA449.
Transmit clock is normally supplied by the external modem.
When an SSI card is connected by a null modem cable,
an on-board clock synthesizer is used.
When testing an SSI card with a null modem cable, configure it
with
lmcconfig(8):
"lmcconfig lmc0 -E -f 10000000"
-E
puts the card in DCE mode to source a transmit clock.
``-f 10000000
''
sets the internal clock source to 10 Mb/s.
T1/E1
A T1 null modem cable has two twisted pairs that connect
pins 1 and 2 on one plug to pins 4 and 5 on the other plug.
Looking into the cable entry hole of a plug,
with the locking tab oriented down,
pin 1 is on the left.
A twisted pair Ethernet cable makes an excellent straight T1 cable.
Alas, Ethernet cross-over cables do not work as T1 null modem cables.
OPERATION NOTES
Packet Lengths
Maximum transmit and receive packet length is unlimited.
Minimum transmit and receive packet length is one byte.
Cleaning up after one packet and setting up for the next
packet involves making several DMA references.
This can take longer than the duration of a short packet,
causing the adapter to fall behind.
For typical PCI bus traffic levels and memory system latencies,
back-to-back packets longer than about 20 bytes will always
work (53 byte cells work), but a burst of several hundred
back-to-back packets shorter than 20 bytes will cause packets
to be dropped.
This usually is not a problem since an IPv4 packet header is
at least 20 bytes long.
This device driver imposes no constraints on packet size.
Most operating systems set the default Maximum Transmission
Unit (MTU) to 1500 bytes; the legal range is usually (72..65535).
This can be changed with
"ifconfig lmc0 mtu 2000"
SPPP enforces an MTU of (128..far-end-MRU) for PPP
and 1500 bytes for Cisco-HDLC.
RAWIP sets the default MTU to 4032 bytes,
but it can be changed to anything.
BPF - Berkeley Packet Filter
This driver has hooks for
bpf(4),
the Berkeley Packet Filter.
The line protocol header length reported to BPF is four bytes
for SPPP and P2P line protocols and zero bytes for RawIP.
To include BPF support into your kernel,
add the following line to
conf/YOURKERNEL
"device bpf"
To test the BPF kernel interface,
bring up a link between two machines, then run
ping(8)
and
tcpdump(1):
Line protocol control packets will appear among the
ping(8)
packets occasionally.
Device Polling
A T3 receiver can generate over 100K interrupts per second,
This can cause a system to
``live-lock''
spend all of its
time servicing interrupts.
Fx has a polling mechanism to prevent live-lock.
Fx Ns 's
mechanism permanently disables interrupts from the card
and instead the card's interrupt service routine is called each
time the kernel is entered (syscall, timer interrupt, etc.) and
from the kernel idle loop; this adds some latency.
The driver is permitted to process a limited number of packets.
The percentage of the CPU that can be consumed this way is settable.
See the
polling(4)
manpage for details on how to enable the polling mode.
SNMP: Simple Network Management Protocol
This driver is aware of what is required to be a Network Interface
Object managed by an Agent of the Simple Network Management Protocol.
The driver exports SNMP-formatted configuration and status
information sufficient for an SNMP Agent to create MIBs for:
"RFC-2233: Interfaces group" ,
"RFC-2496: DS3 interfaces" ,
"RFC-2495: DS1/E1 interfaces" ,
"RFC-1659: RS232-like interfaces" .
An SNMP Agent is a user program, not a kernel function.
Agents can retrieve configuration and status information
by using
Netgraph control messages or
ioctl(2)
system calls.
User programs should poll
sc->cfg.ticks
which increments once per second after the SNMP state has been updated.
HSSI and SSI LEDs
The card should be operational if all three green LEDs are on
(the upper-left one should be blinking) and the red LED is off.
All four LEDs turn on at power-on and module unload.
"RED" Ta upper-right Ta Transmit clock
"GREEN" Ta upper-left Ta Device driver is alive if blinking
"GREEN" Ta lower-right Ta Modem signals are good
"GREEN" Ta lower-left Ta Cable is plugged in (SSI only)
T1E1 and T3 LEDs
The card should be operational if the upper-left green LED is blinking
and all other LEDs are off.
For the T3 card, if other LEDs are on or
blinking, try swapping the coax cables!
All four LEDs turn on at power-on and module unload.
"RED" Ta upper-right Ta Received signal is wrong
"GREEN" Ta upper-left Ta Device driver is alive if blinking
"BLUE" Ta lower-right Ta Alarm Information Signal (AIS)
"YELLOW" Ta lower-left Ta Remote Alarm Indication (RAI)
"The green" Ta LED blinks if the device driver is alive.
"The red" Ta LED blinks if an outward loopback is active.
"The blue" Ta LED blinks if sending AIS, on solid if receiving AIS.
"The yellow" Ta LED blinks if sending RAI, on solid if receiving RAI.
E1 Framing
Phone companies usually insist that customers put a
Frame Alignment Signal
(FAS) in time slot 0.
A Cyclic Redundancy Checksum (CRC) can also ride in time slot 0.
Channel Associated Signalling
(CAS) uses Time Slot 16.
#include <telco-speak> signalling
is on/off hook, ringing, busy, etc.
Signalling is not needed here and consumes 64 Kb/s.
Only use E1-CAS formats if the other end insists on it!
Use E1-FAS+CRC framing format on a public circuit.
Depending on the equipment installed in a private circuit,
it may be possible to use all 32 time slots for data (E1-NONE).
T3 Framing
M13 is a technique for multiplexing 28 T1s into a T3.
Muxes use the C-bits for speed-matching the tributaries.
Muxing is not needed here and usurps the FEBE and FEAC bits.
Only use T3-M13 format if the other end insists on it!
Use T3-CParity framing format if possible.
Loop Timing, Fractional T3, and HDLC packets in
the Facility Data Link are
not
supported.
T1 & T3 Frame Overhead Functions
Performance Report Messages (PRMs) are enabled in T1-ESF.
Bit Oriented Protocol (BOP) messages are enabled in T1-ESF.
In-band loopback control (framed or not) is enabled in T1-SF.
Far End Alarm and Control (FEAC) msgs are enabled in T3-CPar.
Far End Block Error (FEBE) reports are enabled in T3-CPar.
Remote Alarm Indication (RAI) is enabled in T3-Any.
Loopbacks initiated remotely time out after 300 seconds.
T1/E1 'Fractional' 64 kb/s Time Slots
T1 uses time slots 24..1; E1 uses time slots 31..0.
E1 uses TS0 for FAS overhead and TS16 for CAS overhead.
E1-NONE has
no
overhead, so all 32 TSs are available for data.
Enable/disable time slots by setting 32 1s/0s in a config param.
Enabling an E1 overhead time slot,
or enabling TS0 or TS25-TS31 for T1,
is ignored by the driver, which knows better.
The default TS param, 0xFFFFFFFF, enables the maximum number
of time slots for whatever frame format is selected.
56 Kb/s time slots are
not
supported.
T1 Raw Mode
Special gate array microcode exists for the T1/E1 card.
Each T1 frame of 24 bytes is treated as a packet.
A raw T1 byte stream can be delivered to main memory
and transmitted from main memory.
The T1 card adds or deletes framing bits but does not
touch the data.
ATM cells can be transmitted and received this way, with
the software doing all the work.
But that is not hard; after all it is only 1.5 Mb/s second!
T3 Circuit Emulation Mode
Special gate array microcode exists for the T3 card.
Each T3 frame of 595 bytes is treated as a packet.
A raw T3 signal can be
packetized
transported through a
packet network (using some protocol) and then
reconstituted
as a T3 signal at the far end.
The output transmitter's
bit rate can be controlled from software so that it can be
frequency locked
to the distant input signal.
HSSI and SSI Transmit Clocks
Synchronous interfaces use two transmit clocks to eliminate
skew
caused by speed-of-light delays in the modem cable.
DCEs (modems) drive ST, Send Timing, the first transmit clock.
DTEs (hosts) receive ST and use it to clock transmit data, TD,
onto the modem cable.
DTEs also drive a copy of ST back towards the DCE and call it TT,
Transmit Timing, the second transmit clock.
DCEs receive TT and TD and use TT to clock TD into a flip flop.
TT experiences the same delay as (and has no
skew
relative to) TD.
Thus, cable length does not affect data/clock timing.
An Ron Crane
had the idea to use a Fast Ethernet chip as a PCI interface
and add an Ethernet-to-HDLC gate array to make a WAN card.
An David Boggs
designed the Ethernet-to-HDLC gate array and PC cards.
We did this at our company, LAN Media Corporation
(LMC)SBE
Corp. acquired
LMC
and continues to make the cards.
Since the cards use Tulip Ethernet chips, we started with
An Matt Thomas Ns '
ubiquitous
de(4)
driver.
An Michael Graff
stripped out the Ethernet stuff and added HSSI stuff.
An Basil Gunn
ported it to
Solaris
(lost) and
Rob Braun
ported it to
Linux
An Andrew Stanley-Jones
added support
for three more cards and wrote the first version of
lmcconfig(8).
An David Boggs
rewrote everything and now feels responsible for it.
