Server configurations
Server configurationsThese pages describe the server configurations, that are required and provided by IP6M.
The IP6M distribution contains an Ansible playbook, that installs everything on a freshly installed, "empty" Ubutnu 19.04 server.
Ubuntu 19.04 version required mainly because it contains ISC Dhcpd 4.4, that has better IPv6 support than version 4.3 in Ubuntu 18.04 LTS version.
Also, current Drupal 8 requires PHP 7.2, which is available in Ubuntu 19.04.
Drupal
DrupalWhy Drupal?
Drupal version 8 is a leading content management platform, that has the following key features:
- Content entity subsytem
- User management with role, permission management
- Configuration management
- Rest API
- Themeable user interface
- Open sorurce
- Large number of contributed extensions, big developer community.
- Command line tools
Architecture
IP6M installs Drupal instance under
/var/www/drupal8
with Mysql database. Installation is done with Composer and Drush.
IP6M extensions for Drupal are the following:
- Modules
- ip6m_core
- ip6m_dhcp
- ip6m_docsis
- ip6m_voip
- ip6m_testdata
- Frontend App
- Theme
- Drupal configuration
Modules, Frontend app, theme and configuration has their separated repositories.
Frontend App is based on Vue.JS and Typescript, and has multiple use cases. It is a Drupal module that provides some user interface functions, and can be compiled with Apache Cordova as a standalone mobile application interface to IP6M.
Web server
Web serverIP6M uses Apache http server and PHP-fpm. Configurations are on the following pages.
Apache web server
Apache web serverRequired Apache modules:
- rewrite
- headers
- proxy_http
- proxy_fcgi
- proxy_wstunnel
The Apache2 virtual hosting configuration,
/etc/apache2/sites-available/01-drupal8.conf
- created by the Ansible playbook from template - is the following:
<virtualhost> ServerName test.ip6m.net ServerAdmin info@ip6m.net DocumentRoot /var/www/drupal8/web ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined <directory drupal8="" var="" www="">; AllowOverride All SetEnvIf Origin "http(s)?://(mobile\.)?(test.ip6m.net|localhost:8080)$" AccessControlAllowOrigin=$0 Header add Access-Control-Allow-Origin %{AccessControlAllowOrigin}e env=AccessControlAllowOrigin Header add Access-Control-Allow-Credentials true Header add Access-Control-Allow-Headers "x-csrf-token, authorization, content-type, accept, origin, x-requested-with, access-control-allow-origin, x-allowed-header, *" Header add Access-Control-Allow-Methods: * </directory>; RewriteEngine On RewriteCond %{REQUEST_URI} ^/socket.io [NC] RewriteCond %{QUERY_STRING} transport=websocket [NC] RewriteRule /(.*) ws://localhost:8082/$1 [P,L] ProxyPass /socket.io http://localhost:8082/socket.io ProxyPassReverse /socket.io http://localhost:8082/socket.io </virtualhost>
The following section works as reverse http proxy that separates the Drupal http and the NodeJS server's websocket traffic:
RewriteEngine On RewriteCond %{REQUEST_URI} ^/socket.io [NC] RewriteCond %{QUERY_STRING} transport=websocket [NC] RewriteRule /(.*) ws://localhost:8082/$1 [P,L] ProxyPass /socket.io http://localhost:8082/socket.io ProxyPassReverse /socket.io http://localhost:8082/socket.io
The SSL version of the configuration has the following additional lines, expecting that a Letsencrypt https certicifated is installed:
SSLEngine on SSLCertificateFile /etc/letsencrypt/live/test.ip6m.net/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/test.ip6m.net/privkey.pem
PHP
PHPThe required PHP version for Drupal 8 is 7.2.
The required modules:
- php-mysql
- php-fpm
- php-xml
- php-mbstring
- php-gd
- php7.2-xml
- php-curl
The IP6M installer playbook installs and configure php-fpm (FastCGI Process Manager) for higher performance and optimal memory usage.
Dhcp
DhcpDHCP (Dynamic Host Configuration Protocol) is a network management protocol for assigning IP addresses and other configurations to hosts on the network.
IP6M DHCP server function is provided by the ISC Dhcpd v4.4 package, found in the Ubuntu Linux distribution. It has two separated process for IPv4 and IPv6, differentiated by the -4 and -6 command line options.
The two processes listed by the "ps" command:
dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf /etc/dhcp/dhcpd.conf dhcpd -user dhcpd -group dhcpd -f -6 -pf /run/dhcp-server/dhcpd6.pid -cf /etc/dhcp/dhcpd6.conf
The two server can be controlled independently by the following commands:
# service isc-dhcp-server [start|stop|restart] # service isc-dhcp-server6 [start|stop|restart]
The next pages show the detailed configurations.
Dhcp Version 4
Dhcp Version 4The DHCP version 4 process is responsible for IPv4 address configuration for cable modems, MTA-s and CPE devices. IPv4 addressing still necesery for all devices, because most of the Internet uses only IPv4 nowadays. Actual dhcpd.conf can be checked under "/ip6m_dhcp/dhcpd" url of a working IP6M instance.
The configuration file has the following sections:
- header, global options
- dhcp classes
- subnets
- hosts
Global options
option option-122 code 122 = string; next-server 192.168.89.10; omapi-port 7911; log-facility local7; log(info, concat("LEASE LOG|", pick-first-value(binary-to-ascii(10, 8, ".", leased-address), ""), "|", binary-to-ascii(16, 8, ":", option agent.remote-id), "|", binary-to-ascii(16, 8, ":", option dhcp-client-identifier))); option option-120 code 120 = unsigned integer 32; option option-120 0; option log-servers 192.168.89.10; option tftp-server-name "192.168.89.10"; option space packetcable; option packetcable.primary-dhcp-server code 1 = ip-address; option packetcable.secondary-dhcp-server code 2 = ip-address; option packetcable.provisioning-server code 3 = { boolean, ip-address }; option packetcable.as-request-timers code 4 = { unsigned integer 32, unsigned integer 32, unsigned integer 32 }; option packetcable.ap-request-timers code 5 = { unsigned integer 32, unsigned integer 32, unsigned integer 32 }; option packetcable.kerberos-realm-name code 6 = text; option packetcable.kerberos-use-tgt code 7 = boolean; option packetcable.provisioning-timer code 8 = unsigned integer 8; option packetcable.new-provisioning-server code 15 = { boolean, ip-address }; option packetcable-encapsulation code 122 = encapsulate packetcable; option option-66 code 66 = string; option packetcable.primary-dhcp-server 192.168.89.10; option packetcable.secondary-dhcp-server 192.168.89.10; option packetcable.provisioning-server true 192.168.89.10; option packetcable.kerberos-realm-name "\005BASIC\0011\000"; option packetcable.kerberos-use-tgt false; option time-servers 192.168.89.10; option time-offset 0; option domain-name "ip6m.net"; use-host-decl-names on;
Most of these lines define packetcable.* options (option-122 dhcp option), which is required for docsis voip mta configuration.
192.168.89.10 is a sample IPv4 address of the IP6M server.
DHCP Classes
Dhcp classes needed when ip address assignment cannot be static based on the client's mac address, because it doesn't known, but it has to be done based on other information, for example the dhcp forwarding agent that sent the dhcp request to the server. This is the typical situation when the server has to assign IP address to an unknown mac address behind a known cable modem.
An example of such a class is:
class "CMTS-1" { match if ( (option agent.remote-id = 44:32:C8:21:9:E7) and (option dhcp-client-identifier != 1:44:32:C8:21:09:E8) and (option dhcp-client-identifier != ff:c8:21:09:e8:0:3:0:1:44:32:c8:21:9:e8) or (option agent.remote-id = 44:32:C8:41:8C:67) and (option dhcp-client-identifier != 1:44:32:C8:41:8C:68) and (option dhcp-client-identifier != ff:c8:41:8c:68:0:3:0:1:44:32:c8:41:8c:68) or (option agent.remote-id = 0:1E:6B:E6:B4:A5) and (option dhcp-client-identifier != 1:00:1E:6B:E6:B4:A6) and (option dhcp-client-identifier != ff:6b:e6:b4:a6:0:3:0:1:0:1e:6b:e6:b4:a6) or (option agent.remote-id = C:EE:E6:E5:35:2C) and (option dhcp-client-identifier != 1:0C:EE:E6:E5:35:2D) and (option dhcp-client-identifier != ff:e6:e5:35:2d:0:3:0:1:c:ee:e6:e5:35:2d) or (option agent.remote-id = 90:6E:BB:54:F0:D0) or (option agent.remote-id = 0:11:E6:F2:2C:94) and (option dhcp-client-identifier != 1:00:11:E6:F2:2C:95) and (option dhcp-client-identifier != ff:e6:f2:2c:95:0:3:0:1:0:11:e6:f2:2c:95) ); }
Let's analyze the first row:
(option agent.remote-id = 44:32:C8:21:9:E7) and (option dhcp-client-identifier != 1:44:32:C8:21:09:E8) and (option dhcp-client-identifier != ff:c8:21:09:e8:0:3:0:1:44:32:c8:21:9:e8)
A dhcp request is matched if its forwarding agent is "44:32:C8:21:9:E7", and its dhcp client id (until Docsis 2.0 mac address) is not (!=) one of the other two identifiers. In this example the other two identifier is the ID of the voip MTA in the cable modem, so only other mac addresses will match in this class. The MTA will get it's IP address in an other class or static assignment.
Subnets
A typical subnet declaration with dynamic ip assigment to CPE devices based on the class above is the following:
subnet 10.2.0.0 netmask 255.255.0.0 { option subnet-mask 255.255.0.0; option broadcast-address 10.2.255.255; option routers 10.2.0.1; option time-servers 192.168.89.10; option time-offset 7200; option domain-name-servers 8.8.8.8, 8.8.4.4; max-lease-time 7200; default-lease-time 7200; server-identifier 192.168.89.10; pool { allow members of "CMTS-1"; range 10.2.0.100 10.2.255.254; } }
Of course, every class and other networks need to have their own subnet declaration. A minimal subnet declaration like this:
subnet 192.168.89.0 netmask 255.255.255.0 { }
is needed for the server local network, even if we don't want to server any clients in this subnet.
Hosts
Docsis cable modems, MTA-s have their own host records as follows:
host modem-2 { hardware ethernet 44:32:c8:21:09:e7; fixed-address 10.1.0.3; filename "cm/cm-2.cfg"; } host modem-3 { hardware ethernet 44:32:c8:41:8c:67; fixed-address 10.1.0.4; filename "cm/cm-3.cfg"; } host modem-4 { hardware ethernet 00:1e:6b:e6:b4:a5; fixed-address 10.1.0.5; filename "cm/cm-4.cfg"; }
These records fairly simple, the request that matches the ethernet mac address gets the ip address defined in the "fixed-address" field and a "filename" option that is required by cable modems, where they find their configuration file to download by TFTP. (See next section)
A complete, working dhcp.conf file is generated by IP6M if you install the "ip6m_testdata" module, or add your own configuration.
Dhcp Version 6
Dhcp Version 6IPv6 DHCP configuration (/etc/dhcp/dhcpd6.conf) has the following sections:
- Global options
- Cablelabs options
- Subnets
- Hosts
Global options provided by the Dhcpd distribution package. IP6M adds its own data via include:
include "/etc/dhcp/ip6m-dhcpd6.conf";
at the end of the default dhcpd6.conf file. The content of this file is the following. A complete, working example of this config file can be found in IP6M under the " /ip6m_dhcp/dhcpd6" path if the "ip6m_testdata" module is enabled.
Cablelabs options definition
# declare the option space where the CableLabs options live option space docsis code width 2 length width 2 hash size 100; option docsis.tftp-servers code 32 = array of ip6-address; option docsis.configuration-file code 33 = text; option docsis.syslog-servers code 34 = array of ip6-address; option docsis.device-id code 36 = string; option docsis.time-servers code 37 = array of ip6-address; option docsis.time-offset code 38 = signed integer 32; option vsio.docsis code 4491 = encapsulate docsis;
Subnets
Similar to the IPv4 configuration, a subnet declaration is needed for the server's local network. This assumes, that the server has its IPv6 address assigned from the 2001:db8::/64 network, e.g.: 2001:db8::10/64
subnet6 2001:db8::/64 { range6 2001:db8::100 2001:db8::110; }
Subnet declaration for a network behind a Docsis CMTS with IPv6 prefix delegating:
subnet6 2001:db8:4:11::/64 { range6 2001:db8:4:11::2 2001:db8:4:11::100; prefix6 2001:db8:4:100:: 2001:db8:4:200:: /64; option docsis.syslog-servers 2001:db8::10; option docsis.time-servers 2001:db8::10; option docsis.tftp-servers 2001:db8::10; option docsis.time-offset 3600; }
Hosts in this subnet receive IP address from range 2001:db8:4:11::2 -> 2001:db8:4:11::100 and a delegated /64 prefix from the 2001:db8:4:100:: to 2001:db8:4:200:: range. This prefix is used in the customer's local network to assign unique, public IP addresses to every host. This is the main difference between IPv4 and IPv6 nowadays, when IPv4 networks use private ip addresses translated to the one public address assigned to the CPE router.
Hosts
host modem-2 { hardware ethernet 44:32:c8:21:09:e7; fixed-address6 2001:db8:4:11::4432:c821:09e7; option docsis.configuration-file "cm/cm-2.cfg"; } host modem-3 { hardware ethernet 44:32:c8:41:8c:67; fixed-address6 2001:db8:4:11::4432:c841:8c67; option docsis.configuration-file "cm/cm-3.cfg"; } host modem-4 { hardware ethernet 00:1e:6b:e6:b4:a5; fixed-address6 2001:db8:4:11::001e:6be6:b4a5; option docsis.configuration-file "cm/cm-4.cfg"; }
Host declarations for Docsis cable modems are very similar to the IPv4 version. IP6M calculates fix IP addresses from the cable modem's mac address.
Xinetd
XinetdIP6M uses Xinetd for some services needed by DOCSIS specification. The installer enables the following services:
- UDP Time Server
- Atftpd daemon
After installing xinetd package:
apt install xinetd
time server can be enabled in the
/etc/xinetd.d/time
file at the UDP section:
# This is the udp version. service time { disable = no type = INTERNAL id = time-dgram socket_type = dgram protocol = udp user = root wait = yes }
setting the "disable" variable to "no" and restarting the xinetd service:
$ sudo service xinetd restart
Service can be tested with the "rdate" command:
$ sudo apt install rdate $ sudo rdate -u localhost Tue Oct 8 20:42:34 CEST 2019
Rdate has to return current server time.
Tftp
TftpIP6M uses Atftpd package for tftp service. Since Atftpd doesn't support IPv6, we need Xinetd to run it.
Installing atftpd:
sudo apt install atftpd
Xinetd configuration (/etc/xinetd.d/tftp) for Atftpd is the following:
service tftp { socket_type = dgram protocol = udp port = 69 wait = yes user = www-data server = /usr/sbin/in.tftpd server_args = /var/www/drupal8/private/tftp disable = no }
Most interesting option is the "server_args" option, that points to the private files directory under the Drupal instance. This is where the Drupal application creates the cable modem configuration files, and atftpd serves them to cable modems.
DNS relay
DNS relayIP6M uses PowerDNS for dns recursor server. Installing the pacckage:
apt install pdns-recursor
The configuration file is at:
/etc/powerdns/recursor.conf
The options that has to be adjusted:
allow-from= 192.168.0.0/16, 2001:db8::/32
Our IP address ranges where DNS querys come from.
local-address= 192.168.89.10, 127.0.0.1, ::1, 2001:db8::10
The IP addresses of the server.
max-cache-entries= 100000Dns cache size. The default 1M may require too much memory.
query-local-address6= 2001:db8::10
This is necessery for PowerDNS to originate outgoing DNS queries from its IPv6 address.
IP6M-nodejs server
IP6M-nodejs serverIP6M has a NodeJS based daemon component which provides websocket interface to frontend clients. Websocket creates permanent connection from the web client to the server, allowing bi-directional data communication on a single TCP channel. This is very useful for near real-time data visualisation because the client doesn't have to poll the server with expensive http requests frequently. IP6M uses this mechanism for RRD data graphing and other functions.
The NodeJS server is installed under
/var/www/drupal8/ip6m-nodejs
directory.
The Systemd startup file, that is created by the Ansible plyabook is the following:
[Unit] Description=IP6M-nodejs Documentation=https://ip6m.net/documentation After=network.target [Service] Environment=DRUPAL_URL=https://test.ip6m.net/ Environment=NODE_PORT=8082 Environment=RRD_DIRECTORY=/var/www/drupal8/private/rrdbot/db Environment=TIMEZONE_OFFSET=0 Type=simple User=www-data WorkingDirectory=/var/www/drupal8/ip6m-nodejs StandardOutput=file:/var/log/ip6m-nodejs.sys.log StandardError=file:/var/log/ip6m-nodejs.error.log ExecStart=/usr/local/bin/yarn serve Restart=on-failure [Install] WantedBy=multi-user.target
The Server can be controlled by the "service" command:
$ sudo service ip6m-nodejs [start|stop|restart|status]
RRDBot
RRDBotIP6M uses RRDBot SNMP polling daemon to collect statistical data from network devices e.g. cable modems, cmtses. It is able to poll large number of devices simultaniously in a very frequent timing. IP6M uses 10s timing, allowing near "real-time" statistics. RRDBot stores collected data in RRDTool (Round Robin Database) format files.
RRDBot is a C program, the IP6M installer playbook downloads the source code, compile RRDBot and creates Systemd startup configuration for it:
/lib/systemd/system/rrdbotd.service
Content for this file is:
[Unit] Description=rrdbotd Documentation=https://github.com/REANNZ/rrdbot After=network.target [Service] Type=simple User=www-data WorkingDirectory=/var/www/drupal8/private/rrdbot/conf StandardOutput=file:/var/log/rrdbot.sys.log StandardError=file:/var/log/rrdbot.error.log ExecStart=/usr/local/sbin/rrdbotd -c /var/www/drupal8/private/rrdbot/conf Restart=on-failure [Install] WantedBy=multi-user.target
Commands provided by RRDBot
rrdbot-create [-c config directory]
This command creates RRD database files according to the RRDBot device config files.
A sample RRDBot config file for a Docsis Cable modem is the following:
[general] rrd: /var/www/drupal8/private/rrdbot/db/cablemodem-2.rrd [poll] interval: 10 in.source: snmp2c://public@10.1.0.3/ifInOctets.2 out.source: snmp2c://public@10.1.0.3/ifOutOctets.2 dspwr.source: snmp2c://public@10.1.0.3/docsIfDownChannelPower.3 uspwr.source: snmp2c://public@10.1.0.3/docsIfCmStatusTxPower.2 dssnr.source: snmp2c://public@10.1.0.3/docsIfSigQSignalNoise.3 microref.source: snmp2c://public@10.1.0.3/docsIfSigQMicroreflections.3 corrected.source: snmp2c://public@10.1.0.3/docsIfSigQCorrecteds.3 uncorrected.source: snmp2c://public@10.1.0.3/docsIfSigQUncorrectables.3 [create] in.type: COUNTER out.type: COUNTER dspwr.type: GAUGE uspwr.type: GAUGE dssnr.type: GAUGE microref.type: GAUGE corrected.type: COUNTER uncorrected.type: COUNTER usrcv.type: GAUGE ussnr.type: GAUGE cf: AVERAGE archive: 6/minute * 2 days, 30/hour * 1 month, 2/hour * 1 year
IP6M takes care of creating these config files and calling rrdbot-create in case of a cable modem has been created or updated.
rrdbot-get snmp://community@host/oid
This command can be used to manually test an snmp request.
rrdbotd
This is the daemon.
Docsis config compiler
Docsis config compilerDocsis cable modem configuration compiler called "docsis" is an open source program, that can be downloaded from https://github.com/rlaager/docsis/
IP6M uses this utility to compile generated text format config files into binary format that can be downloaded by cable modems via tftp.