Documentation

Documentation

IP6M is a complete server solution that contains all software components and an automated, Ansible based installer.

thomas Wed, 10/02/2019 - 22:04

Requirements

Requirements

These are the minimum requirements to start using an IP6M server.

  • 4GB Ram (8GB recommended)
  • 100GB disk space, SSD recommended
  • Ubuntu 19.04 "disco" operating system with Python package installed
  • Linux user with sudo root permission

In a production environment  requirements highly depends on the network size, the number of the concurrent user or API sessions, the verbosity of logging, and other factors.

 

thomas Fri, 10/04/2019 - 12:05

Server configurations

Server configurations

These pages describe the server configurations, that are required and provided by IP6M.

The IP6M distribution contains an Ansible playbook, that installs everything on a freshly installed, "empty" Ubutnu 19.04 server.

Ubuntu 19.04 version required mainly because it contains ISC Dhcpd 4.4, that has better IPv6 support than version 4.3 in Ubuntu 18.04 LTS version.

Also, current Drupal 8 requires PHP 7.2, which is available in Ubuntu 19.04.

 

thomas Fri, 10/04/2019 - 11:28

Drupal

Drupal

Why Drupal?

 

Drupal version 8 is a leading content management platform, that has the following key features:

  • Content entity subsytem
  • User management with role, permission management
  • Configuration management
  • Rest API
  • Themeable user interface
  • Open sorurce
  • Large number of contributed extensions, big developer community.
  • Command line tools

 

Architecture

 

IP6M installs Drupal instance under

/var/www/drupal8

with Mysql database. Installation is done with Composer and Drush.

IP6M extensions for Drupal are the following:

  • Modules
    • ip6m_core
    • ip6m_dhcp
    • ip6m_docsis
    • ip6m_voip
    • ip6m_testdata
  • Frontend App
  • Theme
  • Drupal configuration

Modules, Frontend app, theme and configuration has their separated repositories.

Frontend App is based on Vue.JS and Typescript, and has multiple use cases. It is a Drupal module that provides some user interface functions, and can be compiled with Apache Cordova as a standalone mobile application interface to IP6M.

 

thomas Fri, 10/04/2019 - 16:07

Web server

Web server

IP6M uses Apache http server and PHP-fpm. Configurations are on the following pages.

thomas Fri, 10/04/2019 - 16:43

Apache web server

Apache web server

Required Apache modules:

  • rewrite
  • headers
  • proxy_http
  • proxy_fcgi
  • proxy_wstunnel

 

The Apache2 virtual hosting configuration,

/etc/apache2/sites-available/01-drupal8.conf

- created by the Ansible playbook from template - is the following:

 
<virtualhost>

        ServerName test.ip6m.net

        ServerAdmin info@ip6m.net
        DocumentRoot /var/www/drupal8/web

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        <directory drupal8="" var="" www="">;
                AllowOverride All
                SetEnvIf Origin "http(s)?://(mobile\.)?(test.ip6m.net|localhost:8080)$" AccessControlAllowOrigin=$0
                Header add Access-Control-Allow-Origin %{AccessControlAllowOrigin}e env=AccessControlAllowOrigin
                Header add Access-Control-Allow-Credentials true
                Header add Access-Control-Allow-Headers "x-csrf-token, authorization, content-type, accept, origin, x-requested-with, access-control-allow-origin, x-allowed-header, *"
                Header add Access-Control-Allow-Methods: *
        </directory>;

        RewriteEngine On

        RewriteCond %{REQUEST_URI}  ^/socket.io            [NC]
        RewriteCond %{QUERY_STRING} transport=websocket    [NC]
        RewriteRule /(.*)           ws://localhost:8082/$1 [P,L]

        ProxyPass        /socket.io  http://localhost:8082/socket.io
        ProxyPassReverse /socket.io  http://localhost:8082/socket.io

</virtualhost>

The following section works as reverse http proxy that separates the Drupal http and the NodeJS server's websocket traffic:

 
       RewriteEngine On

        RewriteCond %{REQUEST_URI}  ^/socket.io            [NC]
        RewriteCond %{QUERY_STRING} transport=websocket    [NC]
        RewriteRule /(.*)           ws://localhost:8082/$1 [P,L]

        ProxyPass        /socket.io  http://localhost:8082/socket.io
        ProxyPassReverse /socket.io  http://localhost:8082/socket.io

The SSL version of the configuration has the following additional lines, expecting that a Letsencrypt https certicifated is installed:

 
SSLEngine on
SSLCertificateFile      /etc/letsencrypt/live/test.ip6m.net/fullchain.pem
SSLCertificateKeyFile   /etc/letsencrypt/live/test.ip6m.net/privkey.pem
thomas Fri, 10/04/2019 - 17:14

PHP

PHP

The required PHP version for Drupal 8 is 7.2.

The required modules:

  • php-mysql
  • php-fpm
  • php-xml
  • php-mbstring
  • php-gd
  • php7.2-xml
  • php-curl

 

The IP6M installer playbook installs and configure php-fpm (FastCGI Process Manager) for higher performance and optimal memory usage.

 

thomas Fri, 10/04/2019 - 17:15

Dhcp

Dhcp

DHCP (Dynamic Host Configuration Protocol) is a network management protocol for assigning IP addresses and other configurations to hosts on the network.

IP6M DHCP server function is provided by the ISC Dhcpd v4.4 package, found in the Ubuntu Linux distribution. It has two separated process for IPv4 and IPv6, differentiated by the -4 and -6 command line options.

The two processes listed by the "ps" command:

dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf /etc/dhcp/dhcpd.conf
dhcpd -user dhcpd -group dhcpd -f -6 -pf /run/dhcp-server/dhcpd6.pid -cf /etc/dhcp/dhcpd6.conf

The two server can be controlled independently by the following commands:

# service isc-dhcp-server [start|stop|restart]

# service isc-dhcp-server6 [start|stop|restart]

The next pages show the detailed configurations.

thomas Fri, 10/04/2019 - 16:08

Dhcp Version 4

Dhcp Version 4

The DHCP version 4 process is responsible for IPv4 address configuration for cable modems, MTA-s and CPE devices. IPv4 addressing still necesery for all devices, because most of the Internet uses only IPv4 nowadays. Actual dhcpd.conf can be checked under "/ip6m_dhcp/dhcpd" url of a working IP6M instance.

The configuration file has the following sections:

  1. header, global options
  2. dhcp classes
  3. subnets
  4. hosts

 

Global options

option option-122 code 122 = string;
next-server 192.168.89.10;
omapi-port 7911;
log-facility local7;
log(info, concat("LEASE LOG|", pick-first-value(binary-to-ascii(10, 8, ".", leased-address), ""), "|", binary-to-ascii(16, 8, ":", option agent.remote-id), "|", binary-to-ascii(16, 8, ":", option dhcp-client-identifier)));
option option-120 code 120 = unsigned integer 32;
option option-120 0;
option log-servers 192.168.89.10;
option tftp-server-name "192.168.89.10";
option space packetcable;
option packetcable.primary-dhcp-server code 1 = ip-address;
option packetcable.secondary-dhcp-server code 2 = ip-address;
option packetcable.provisioning-server code 3 = { boolean, ip-address };
option packetcable.as-request-timers code 4 = { unsigned integer 32, unsigned integer 32, unsigned integer 32 };
option packetcable.ap-request-timers code 5 = { unsigned integer 32, unsigned integer 32, unsigned integer 32 };
option packetcable.kerberos-realm-name code 6 = text;
option packetcable.kerberos-use-tgt code 7 = boolean;
option packetcable.provisioning-timer code 8 = unsigned integer 8;
option packetcable.new-provisioning-server code 15 = { boolean, ip-address };
option packetcable-encapsulation code 122 = encapsulate packetcable;
option option-66 code 66 = string;
option packetcable.primary-dhcp-server 192.168.89.10;
option packetcable.secondary-dhcp-server 192.168.89.10;
option packetcable.provisioning-server true 192.168.89.10;
option packetcable.kerberos-realm-name "\005BASIC\0011\000";
option packetcable.kerberos-use-tgt false;
option time-servers 192.168.89.10;
option time-offset 0;
option domain-name "ip6m.net";
use-host-decl-names on; 

Most of these lines define packetcable.* options (option-122 dhcp option), which is required for docsis voip mta configuration.

192.168.89.10 is a sample IPv4 address of the IP6M server.

 

DHCP Classes

 

Dhcp classes needed when ip address assignment cannot be static based on the client's mac address, because it doesn't known, but it has to be done based on other information, for example the dhcp forwarding agent that sent the dhcp request to the server. This is the typical situation when the server has to assign IP address to an unknown mac address behind a known cable modem.

An example of such a class is:

class "CMTS-1" {
  match if (
    (option agent.remote-id = 44:32:C8:21:9:E7) and (option dhcp-client-identifier != 1:44:32:C8:21:09:E8) and (option dhcp-client-identifier != ff:c8:21:09:e8:0:3:0:1:44:32:c8:21:9:e8) or
    (option agent.remote-id = 44:32:C8:41:8C:67) and (option dhcp-client-identifier != 1:44:32:C8:41:8C:68) and (option dhcp-client-identifier != ff:c8:41:8c:68:0:3:0:1:44:32:c8:41:8c:68) or
    (option agent.remote-id = 0:1E:6B:E6:B4:A5) and (option dhcp-client-identifier != 1:00:1E:6B:E6:B4:A6) and (option dhcp-client-identifier != ff:6b:e6:b4:a6:0:3:0:1:0:1e:6b:e6:b4:a6) or
    (option agent.remote-id = C:EE:E6:E5:35:2C) and (option dhcp-client-identifier != 1:0C:EE:E6:E5:35:2D) and (option dhcp-client-identifier != ff:e6:e5:35:2d:0:3:0:1:c:ee:e6:e5:35:2d) or
    (option agent.remote-id = 90:6E:BB:54:F0:D0) or
    (option agent.remote-id = 0:11:E6:F2:2C:94) and (option dhcp-client-identifier != 1:00:11:E6:F2:2C:95) and (option dhcp-client-identifier != ff:e6:f2:2c:95:0:3:0:1:0:11:e6:f2:2c:95)   );
}

Let's analyze the first row:

(option agent.remote-id = 44:32:C8:21:9:E7) and (option dhcp-client-identifier != 1:44:32:C8:21:09:E8) and (option dhcp-client-identifier != ff:c8:21:09:e8:0:3:0:1:44:32:c8:21:9:e8) 

A dhcp request is matched if its forwarding agent is "44:32:C8:21:9:E7", and its dhcp client id (until Docsis 2.0 mac address) is not (!=) one of the other two identifiers. In this example the other two identifier is the ID of the voip MTA in the cable modem, so only other mac addresses will match in this class. The MTA will get it's IP address in an other class or static assignment.

 

Subnets

 

A typical subnet declaration with dynamic ip assigment to CPE devices based on the class above is the following:

subnet 10.2.0.0 netmask 255.255.0.0 {
  option subnet-mask 255.255.0.0;
  option broadcast-address 10.2.255.255;
  option routers 10.2.0.1;
  option time-servers 192.168.89.10;
  option time-offset 7200;
  option domain-name-servers 8.8.8.8, 8.8.4.4;
  max-lease-time 7200;
  default-lease-time 7200;
  server-identifier 192.168.89.10;
  pool {
    allow members of "CMTS-1";
    range 10.2.0.100 10.2.255.254;
  }
}

Of course, every class and other networks need to have their own subnet declaration. A minimal subnet declaration like this:

subnet 192.168.89.0 netmask 255.255.255.0 {
}

is needed for the server local network, even if we don't want to server any clients in this subnet.

 

Hosts

 

Docsis cable modems, MTA-s have their own host records as follows:

host modem-2 {
	hardware ethernet 44:32:c8:21:09:e7;
	fixed-address 10.1.0.3;
	filename "cm/cm-2.cfg";
}
host modem-3 {
	hardware ethernet 44:32:c8:41:8c:67;
	fixed-address 10.1.0.4;
	filename "cm/cm-3.cfg";
}
host modem-4 {
	hardware ethernet 00:1e:6b:e6:b4:a5;
	fixed-address 10.1.0.5;
	filename "cm/cm-4.cfg";
}

These records fairly simple, the request that matches the ethernet mac address gets the ip address defined in the "fixed-address" field and a "filename" option that is required by cable modems, where they find their configuration file to download by TFTP. (See next section)

A complete, working dhcp.conf file is generated by IP6M if you install the "ip6m_testdata" module, or add your own configuration.

 

thomas Mon, 10/07/2019 - 13:48

Dhcp Version 6

Dhcp Version 6

IPv6 DHCP configuration (/etc/dhcp/dhcpd6.conf) has the following sections:

  1. Global options
  2. Cablelabs options
  3. Subnets
  4. Hosts

 

Global options provided by the Dhcpd distribution package. IP6M adds its own data via include:

include "/etc/dhcp/ip6m-dhcpd6.conf";

at the end of the default dhcpd6.conf file. The content of this file is the following. A complete, working example of this config file can be found in IP6M under the " /ip6m_dhcp/dhcpd6" path if the "ip6m_testdata" module is enabled.

Cablelabs options definition

# declare the option space where the CableLabs options live
option space docsis code width 2 length width 2 hash size 100;
option docsis.tftp-servers code 32 = array of ip6-address;
option docsis.configuration-file code 33 = text;
option docsis.syslog-servers code 34 = array of ip6-address;
option docsis.device-id code 36 = string;
option docsis.time-servers code 37 = array of ip6-address;
option docsis.time-offset code 38 = signed integer 32;
option vsio.docsis code 4491 = encapsulate docsis;

Subnets

Similar to the IPv4 configuration, a subnet declaration is needed for the server's local network. This assumes, that the server has its IPv6 address assigned from the 2001:db8::/64 network, e.g.: 2001:db8::10/64

subnet6 2001:db8::/64 {
	range6 2001:db8::100 2001:db8::110;
}

Subnet declaration for a network behind a Docsis CMTS with IPv6 prefix delegating:

subnet6 2001:db8:4:11::/64 {
	range6 2001:db8:4:11::2 2001:db8:4:11::100;
	prefix6 2001:db8:4:100:: 2001:db8:4:200:: /64;
	option docsis.syslog-servers 2001:db8::10;
	option docsis.time-servers 2001:db8::10;
	option docsis.tftp-servers 2001:db8::10;
	option docsis.time-offset 3600;
}

Hosts in this subnet receive IP address from range 2001:db8:4:11::2 -> 2001:db8:4:11::100 and a delegated /64 prefix from the 2001:db8:4:100:: to 2001:db8:4:200:: range. This prefix is used in the customer's local network to assign unique, public IP addresses to every host. This is the main difference between IPv4 and IPv6 nowadays, when IPv4 networks use private ip addresses translated to the one public address assigned to the CPE router.

Hosts

host modem-2 {
	hardware ethernet 44:32:c8:21:09:e7;
	fixed-address6 2001:db8:4:11::4432:c821:09e7;
	option docsis.configuration-file "cm/cm-2.cfg";
}
host modem-3 {
	hardware ethernet 44:32:c8:41:8c:67;
	fixed-address6 2001:db8:4:11::4432:c841:8c67;
	option docsis.configuration-file "cm/cm-3.cfg";
}
host modem-4 {
	hardware ethernet 00:1e:6b:e6:b4:a5;
	fixed-address6 2001:db8:4:11::001e:6be6:b4a5;
	option docsis.configuration-file "cm/cm-4.cfg";
}

Host declarations for Docsis cable modems are very similar to the IPv4 version. IP6M calculates fix IP addresses from the cable modem's mac address.

 

 

thomas Mon, 10/07/2019 - 20:24

Xinetd

Xinetd

IP6M uses Xinetd for some services needed by DOCSIS specification. The installer enables the following services:

  • UDP Time Server
  • Atftpd daemon

 

After installing xinetd  package:

apt install xinetd

time server can be enabled in the 

/etc/xinetd.d/time

file at the UDP section:

# This is the udp version.
service time
{
    disable      = no
    type         = INTERNAL
    id           = time-dgram
    socket_type  = dgram
    protocol     = udp
    user         = root
    wait         = yes
}

setting the "disable" variable to "no" and restarting the xinetd service:

$ sudo service xinetd restart

Service can be tested with the "rdate" command:

$ sudo apt install rdate

$ sudo rdate -u localhost
Tue Oct  8 20:42:34 CEST 2019

Rdate has to return current server time.

 

thomas Fri, 10/04/2019 - 16:10

Tftp

Tftp

IP6M uses Atftpd package for tftp service. Since Atftpd doesn't support IPv6, we need Xinetd to run it.

Installing atftpd:

sudo apt install atftpd

Xinetd configuration (/etc/xinetd.d/tftp) for Atftpd is the following:

service tftp
{
    socket_type  = dgram
    protocol     = udp
    port         = 69
    wait         = yes
    user         = www-data
    server       = /usr/sbin/in.tftpd
    server_args  = /var/www/drupal8/private/tftp
    disable      = no
}

Most interesting option is the "server_args" option, that points to the private files directory under the Drupal instance. This is where the Drupal application creates the cable modem configuration files, and atftpd serves them to cable modems.

 

thomas Fri, 10/04/2019 - 16:09

DNS relay

DNS relay

IP6M uses PowerDNS for dns recursor server. Installing the pacckage:

apt install pdns-recursor

The configuration file is at:

/etc/powerdns/recursor.conf

The options that has to be adjusted:

allow-from= 192.168.0.0/16, 2001:db8::/32 

Our IP address ranges where DNS querys come from.

local-address= 192.168.89.10, 127.0.0.1, ::1, 2001:db8::10

The IP addresses of the server.

max-cache-entries= 100000
Dns cache size. The default 1M may require too much memory.
query-local-address6= 2001:db8::10

This is necessery for PowerDNS to originate outgoing DNS queries from its IPv6 address.

thomas Fri, 10/04/2019 - 16:18

IP6M-nodejs server

IP6M-nodejs server

IP6M has a NodeJS based daemon component which provides websocket interface to frontend clients. Websocket creates permanent connection from the web client to the server, allowing bi-directional data communication on a single TCP channel. This is very useful for near real-time data visualisation because the client doesn't have to poll the server with expensive http requests frequently. IP6M uses this mechanism for RRD data graphing and other functions.

The NodeJS server is installed under

/var/www/drupal8/ip6m-nodejs

directory.

The Systemd startup file, that is created by the Ansible plyabook is the following:

[Unit]
Description=IP6M-nodejs
Documentation=https://ip6m.net/documentation
After=network.target

[Service]
Environment=DRUPAL_URL=https://test.ip6m.net/
Environment=NODE_PORT=8082
Environment=RRD_DIRECTORY=/var/www/drupal8/private/rrdbot/db
Environment=TIMEZONE_OFFSET=0
Type=simple
User=www-data
WorkingDirectory=/var/www/drupal8/ip6m-nodejs
StandardOutput=file:/var/log/ip6m-nodejs.sys.log
StandardError=file:/var/log/ip6m-nodejs.error.log
ExecStart=/usr/local/bin/yarn serve
Restart=on-failure

[Install]
WantedBy=multi-user.target

 

The Server can be controlled by the "service" command:

$ sudo service ip6m-nodejs [start|stop|restart|status]

 

thomas Fri, 10/04/2019 - 16:42

RRDBot

RRDBot

IP6M uses RRDBot SNMP polling daemon to collect statistical data from network devices e.g. cable modems, cmtses. It is able to poll large number of devices simultaniously in a very frequent timing. IP6M uses 10s timing, allowing near "real-time" statistics. RRDBot stores collected data in RRDTool (Round Robin Database) format files.

RRDBot is a C program, the IP6M installer playbook downloads the source code, compile RRDBot and creates Systemd startup configuration for it:

/lib/systemd/system/rrdbotd.service

Content for this file is:

[Unit]
Description=rrdbotd
Documentation=https://github.com/REANNZ/rrdbot
After=network.target

[Service]
Type=simple
User=www-data
WorkingDirectory=/var/www/drupal8/private/rrdbot/conf
StandardOutput=file:/var/log/rrdbot.sys.log
StandardError=file:/var/log/rrdbot.error.log
ExecStart=/usr/local/sbin/rrdbotd -c /var/www/drupal8/private/rrdbot/conf
Restart=on-failure

[Install]
WantedBy=multi-user.target

 

Commands provided by RRDBot

rrdbot-create [-c config directory]

This command creates RRD database files according to the RRDBot device config files.

A sample RRDBot config file for a Docsis Cable modem is the following:

[general]
rrd: /var/www/drupal8/private/rrdbot/db/cablemodem-2.rrd
[poll]
interval: 10
in.source: snmp2c://public@10.1.0.3/ifInOctets.2
out.source: snmp2c://public@10.1.0.3/ifOutOctets.2
dspwr.source: snmp2c://public@10.1.0.3/docsIfDownChannelPower.3
uspwr.source: snmp2c://public@10.1.0.3/docsIfCmStatusTxPower.2
dssnr.source: snmp2c://public@10.1.0.3/docsIfSigQSignalNoise.3
microref.source: snmp2c://public@10.1.0.3/docsIfSigQMicroreflections.3
corrected.source: snmp2c://public@10.1.0.3/docsIfSigQCorrecteds.3
uncorrected.source: snmp2c://public@10.1.0.3/docsIfSigQUncorrectables.3
[create]
in.type: COUNTER
out.type: COUNTER
dspwr.type: GAUGE
uspwr.type: GAUGE
dssnr.type: GAUGE
microref.type: GAUGE
corrected.type: COUNTER
uncorrected.type: COUNTER
usrcv.type: GAUGE
ussnr.type: GAUGE
cf: AVERAGE
archive: 6/minute * 2 days,
         30/hour * 1 month,
         2/hour * 1 year

IP6M takes care of creating these config files and calling rrdbot-create in case of a cable modem has been created or updated.

rrdbot-get snmp://community@host/oid

This command can be used to manually test an snmp request.

rrdbotd

This is the daemon.

 

thomas Fri, 10/04/2019 - 17:13

Docsis config compiler

Docsis config compiler

Docsis cable modem configuration compiler called "docsis" is an open source program, that can be downloaded from https://github.com/rlaager/docsis/

IP6M uses this utility to compile generated text format config files into binary format that can be downloaded by cable modems via tftp.

thomas Wed, 10/09/2019 - 16:17