perfSONAR: Documentation
Appliance Details
Description
This appliance creates instances that are equipped with perfSONAR. Both Haswell and Skylake servers can be used. Desired perfSONAR bundle can be selected. One perfSONAR node with perfsonar-toolkit is created. Optionally, additional perfSONAR nodes can be created. Instances can be connected to any of the neutron networks (sharednet1, sharedwan1, isolated networks created by the users).
Installation of the perfSONAR software stack is defined as a postboot script (user_data) in the template file. Desired changes for tuning can be done in the user_data section. Installation is done on standard CC-CentOS7 image. There are no special requirements for the base OS.
Fundamental host tuning can be seen in /etc/sysctl.conf and /etc/profile.d/perfsonar.sh files. Additional information can be found here .
Initial Configuration
Configure tests through web-ui
- perfSONAR dashboard is available with the URL https://FLOATING_IP_ADDRESS/toolkit.
- A web user should be created to configure tests via web-ui.
[cc@perfsonar ~]$ sudo su -
Last login: Wed Oct 30 20:20:20 UTC 2019 on pts/0
There isn't a perfSONAR Website Administrator defined.
Enter the user whose account you'd like to add. Just hit enter to exit:
New password:
Re-type new password:
Adding password for user perfsonarweb
There isn't a perfSONAR sudo user defined. It is recommended that you create a sudo user and disable SSH root login.
WARNING: Be sure you remember the username and password you create with this script as root will no longer be able to login via SSH once this command completes. If you are using SSH as root now, your session should continue until you logout.
Would you like to create a sudo user and disable root SSH? [yes]
Enter the user whose account you'd like to add. Just hit enter to exit:
[root@perfsonar ~]# Additional information can be found on here .
Also, Creating and Managing Tasks page includes information about configuring tests with pscheduler.
Basic network measurements
Once a stack with 2 perfSONAR instances are created, network measurement tests can be performed as below:
[root@perfsonar-1 ~]# LOCAL_HOST="192.168.201.12"
[root@perfsonar-1 ~]# REMOTE_HOST="192.168.201.14"
[root@perfsonar-1 ~]# pscheduler ping $REMOTE_HOST
192.168.201.14: pScheduler is alive
[root@perfsonar-1 ~]# pscheduler task trace --dest $REMOTE_HOST --source $LOCAL_HOST
Submitting task...
Task URL:
https://192.168.201.12/pscheduler/tasks/5e997239-e4f4-4290-93bc-43e6072ecf32
Running with tool 'traceroute'
Fetching first run...
Next scheduled run:
https://192.168.201.12/pscheduler/tasks/5e997239-e4f4-4290-93bc-43e6072ecf32/runs/440180a9-fb79-4c6c-8fcc-824fab2b897a
Starts 2019-10-31T05:12:46Z (~2 seconds)
Ends 2019-10-31T05:12:54Z (~7 seconds)
Waiting for result...
1 192.168.201.14 0.2 ms
No further runs scheduled.
[root@perfsonar-1 ~]# pscheduler task rtt --dest $REMOTE_HOST --source $LOCAL_HOST
Submitting task...
Task URL:
https://192.168.201.12/pscheduler/tasks/96d7ac95-3535-4d51-8ac7-c3178955a297
Running with tool 'ping'
Fetching first run...
Next scheduled run:
https://192.168.201.12/pscheduler/tasks/96d7ac95-3535-4d51-8ac7-c3178955a297/runs/142d77b7-da4f-4192-8923-2d5d6aea3a46
Starts 2019-10-31T05:13:12Z (~2 seconds)
Ends 2019-10-31T05:13:23Z (~10 seconds)
Waiting for result...
1 192.168.201.14 64 Bytes TTL 64 RTT 0.2430 ms
2 192.168.201.14 64 Bytes TTL 64 RTT 0.1640 ms
3 192.168.201.14 64 Bytes TTL 64 RTT 0.1840 ms
4 192.168.201.14 64 Bytes TTL 64 RTT 0.1850 ms
5 192.168.201.14 64 Bytes TTL 64 RTT 0.1580 ms
0% Packet Loss RTT Min/Mean/Max/StdDev = 0.158000/0.186000/0.243000/0.034000 ms
No further runs scheduled.
[root@perfsonar-1 ~]# pscheduler task simplestream --dest $REMOTE_HOST --source $LOCAL_HOST
Submitting task...
Task URL:
https://192.168.201.12/pscheduler/tasks/397bd755-3ddf-4514-b85e-30cfc3f64870
Running with tool 'simplestreamer'
Fetching first run...
Next scheduled run:
https://192.168.201.12/pscheduler/tasks/397bd755-3ddf-4514-b85e-30cfc3f64870/runs/c1a13a66-467a-4964-a53b-7480f80a5379
Starts 2019-10-31T05:13:49Z (~6 seconds)
Ends 2019-10-31T05:13:54Z (~4 seconds)
Waiting for result...
Dawdled ......... P0D
Text Sent ....... Data sent at 2019-10-31 05:13:49.226337
Text Received ... Data sent at 2019-10-31 05:13:49.226337
Elapsed Time .... PT0.466438S
No further runs scheduled.
[root@perfsonar-1 ~]# pscheduler task throughput --bandwidth 10G --dest $REMOTE_HOST --source $LOCAL_HOST
Submitting task...
Task URL:
https://192.168.201.12/pscheduler/tasks/5059ecc8-4bf0-4589-96e1-02363359733f
Running with tool 'iperf3'
Fetching first run...
Next scheduled run:
https://192.168.201.12/pscheduler/tasks/5059ecc8-4bf0-4589-96e1-02363359733f/runs/f0ea0e36-fb43-4005-b01d-f930d7fc8376
Starts 2019-10-31T05:14:24Z (~7 seconds)
Ends 2019-10-31T05:14:43Z (~18 seconds)
Waiting for result...
* Stream ID 5
Interval Throughput Retransmits Current Window
0.0 - 1.0 9.87 Gbps 4 796.37 KBytes
1.0 - 2.0 9.89 Gbps 0 859.01 KBytes
2.0 - 3.0 9.89 Gbps 0 876.90 KBytes
3.0 - 4.0 9.89 Gbps 0 903.75 KBytes
4.0 - 5.0 9.89 Gbps 0 1.10 MBytes
5.0 - 6.0 9.89 Gbps 0 1.31 MBytes
6.0 - 7.0 9.89 Gbps 0 1.32 MBytes
7.0 - 8.0 9.89 Gbps 0 1.53 MBytes
8.0 - 9.0 9.89 Gbps 0 1.53 MBytes
9.0 - 10.0 9.90 Gbps 0 1.55 MBytes
Summary
Interval Throughput Retransmits
0.0 - 10.0 9.89 Gbps 4
No further runs scheduled.
[root@perfsonar-1 ~]# pscheduler task latency --dest=$REMOTE_HOST --source=$LOCAL_HOST
Submitting task...
Task URL:
https://192.168.201.12/pscheduler/tasks/e48112c1-8b62-4a06-b50a-a91cf665074d
Running with tool 'owping'
Fetching first run...
Next scheduled run:
https://192.168.201.12/pscheduler/tasks/e48112c1-8b62-4a06-b50a-a91cf665074d/runs/868a847f-de89-44d3-bd62-808881dd94b0
Starts 2019-10-31T05:20:13Z (~3 seconds)
Ends 2019-10-31T05:20:34Z (~20 seconds)
Waiting for result...
Run did not complete: Failed
Limit system diagnostics for this run:
Hints:
requester: 192.168.201.12
server: 192.168.201.12
Identified as everybody, local-interfaces
Classified as default, friendlies
Application: Hosts we trust to do everything
Group 1: Limit 'always' passed
Group 1: Want all, 1/1 passed, 0/1 failed: PASS
Application PASSES
Application: Defaults applied to non-friendly hosts
Group 1: Limit 'innocuous-tests' passed
Group 1: Limit 'throughput-default-time' failed: Test is not 'throughput'
Group 1: Limit 'throughput-default-udp' failed: Test is not 'throughput'
Group 1: Limit 'idleex-default' failed: Test is not 'idleex'
Group 1: Want any, 1/4 passed, 3/4 failed: PASS
Application PASSES
Proposal meets limits
Priority set at 0:
Initial priority (Set to 0)
Error:
Process took too long to run.
Diagnostics from 192.168.201.12:
No diagnostics.
Error from 192.168.201.12:
No error.
No further runs scheduled.Network measurements across UC and TACC
In this setup, we have isolated networks on UC and TACC sites. These networks have the same subnet with separate allocation pools. Both isolated networks are stitched through Layer2 circuits . On each site, 2 perfSONAR nodes are created.
- UC: 192.168.201.12, 192.168.201.14
- TACC: 192.168.201.24, 192.168.201.26
Create Networks
CHI@UC
$ openstack network show net-exogeni-perfsonar-1
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | nova |
| created_at | 2019-10-31T03:48:11Z |
| description | VSwitchName=perfsonar |
| dns_domain | None |
| id | a12e3b1f-02c9-457f-8f08-67fdbd9c8286 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | None |
| is_vlan_transparent | None |
| location | None |
| mtu | 1500 |
| name | net-exogeni-perfsonar-1 |
| port_security_enabled | True |
| project_id | d9faac3973a847f1b718fa765fe312e2 |
| provider:network_type | vlan |
| provider:physical_network | exogeni |
| provider:segmentation_id | 3294 |
| qos_policy_id | None |
| revision_number | 2 |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | c2cc881f-c504-469b-843c-f5f01b1d5bb2 |
| tags | |
| updated_at | 2019-10-31T03:55:44Z |
+---------------------------+--------------------------------------+
$ openstack subnet show subnet-net-exogeni-perfsonar-1
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 192.168.201.11-192.168.201.19 |
| cidr | 192.168.201.0/24 |
| created_at | 2019-10-31T03:55:44Z |
| description | |
| dns_nameservers | 130.202.101.37, 130.202.101.6 |
| enable_dhcp | True |
| gateway_ip | 192.168.201.10 |
| host_routes | |
| id | c2cc881f-c504-469b-843c-f5f01b1d5bb2 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| location | None |
| name | subnet-net-exogeni-perfsonar-1 |
| network_id | a12e3b1f-02c9-457f-8f08-67fdbd9c8286 |
| project_id | d9faac3973a847f1b718fa765fe312e2 |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2019-10-31T03:55:44Z |
+-------------------+--------------------------------------+
$ openstack server list -f table -c Name -c Networks
+-------------+------------------------------------------------------+
| Name | Networks |
+-------------+------------------------------------------------------+
| perfsonar-1 | net-exogeni-perfsonar-1=192.168.201.12, 192.5.87.159 |
| ps-0 | net-exogeni-perfsonar-1=192.168.201.14 |
+-------------+------------------------------------------------------+CHI@TACC
$ openstack network show net-exogeni-perfsonar-1
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | nova |
| created_at | 2019-10-31T03:49:02Z |
| description | VSwitchName=perfsonar |
| dns_domain | None |
| id | 1535a613-10f0-4ecd-b2aa-0267a6530b4a |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | None |
| is_vlan_transparent | None |
| location | None |
| mtu | 1500 |
| name | net-exogeni-perfsonar-1 |
| port_security_enabled | True |
| project_id | d9faac3973a847f1b718fa765fe312e2 |
| provider:network_type | vlan |
| provider:physical_network | exogeni |
| provider:segmentation_id | 3509 |
| qos_policy_id | None |
| revision_number | 3 |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | 12b694f5-7532-4bc8-a34b-59e8b11bdf13 |
| tags | |
| updated_at | 2019-10-31T04:26:05Z |
+---------------------------+--------------------------------------+
$ openstack subnet show subnet-net-exogeni-perfsonar-1
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 192.168.201.21-192.168.201.29 |
| cidr | 192.168.201.0/24 |
| created_at | 2019-10-31T03:56:14Z |
| description | |
| dns_nameservers | 8.8.4.4, 8.8.8.8 |
| enable_dhcp | True |
| gateway_ip | 192.168.201.20 |
| host_routes | |
| id | 12b694f5-7532-4bc8-a34b-59e8b11bdf13 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| location | None |
| name | subnet-net-exogeni-perfsonar-1 |
| network_id | 1535a613-10f0-4ecd-b2aa-0267a6530b4a |
| project_id | d9faac3973a847f1b718fa765fe312e2 |
| revision_number | 1 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2019-10-31T04:26:05Z |
+-------------------+--------------------------------------+
$ openstack server list -f table -c Name -c Networks
+-------------+--------------------------------------------------------+
| Name | Networks |
+-------------+--------------------------------------------------------+
| perfsonar-2 | net-exogeni-perfsonar-1=192.168.201.24, 129.114.109.18 |
| ps-0 | net-exogeni-perfsonar-1=192.168.201.26 |
+-------------+--------------------------------------------------------+Network Measurements
[root@perfsonar-1 ~]# pscheduler task rtt --dest $REMOTE_HOST --source $LOCAL_HOST
Submitting task...
Task URL:
https://192.168.201.12/pscheduler/tasks/f20d9626-65d3-482e-9283-0882a3655580
Running with tool 'ping'
Fetching first run...
Next scheduled run:
https://192.168.201.12/pscheduler/tasks/f20d9626-65d3-482e-9283-0882a3655580/runs/323cac7d-8432-4b46-be87-e27e4ab4d56c
Starts 2019-10-31T05:53:05Z (~4 seconds)
Ends 2019-10-31T05:53:16Z (~10 seconds)
Waiting for result...
1 192.168.201.24 64 Bytes TTL 64 RTT 31.2000 ms
2 192.168.201.24 64 Bytes TTL 64 RTT 31.1000 ms
3 192.168.201.24 64 Bytes TTL 64 RTT 31.2000 ms
4 192.168.201.24 64 Bytes TTL 64 RTT 31.1000 ms
5 192.168.201.24 64 Bytes TTL 64 RTT 31.1000 ms
0% Packet Loss RTT Min/Mean/Max/StdDev = 31.122000/31.181000/31.243000/0.164000 ms
No further runs scheduled.
[root@perfsonar-1 ~]# pscheduler task throughput --bandwidth 10G --dest $REMOTE_HOST --source $LOCAL_HOST
Submitting task...
Task URL:
https://192.168.201.12/pscheduler/tasks/f04b87c4-d466-40d0-90db-ded10fb100b7
Running with tool 'iperf3'
Fetching first run...
Next scheduled run:
https://192.168.201.12/pscheduler/tasks/f04b87c4-d466-40d0-90db-ded10fb100b7/runs/c2fae915-6803-465d-9811-a21f16eeef3a
Starts 2019-10-31T05:52:04Z (~7 seconds)
Ends 2019-10-31T05:52:23Z (~18 seconds)
Waiting for result...
* Stream ID 5
Interval Throughput Retransmits Current Window
0.0 - 1.0 7.55 Gbps 6 78.16 MBytes
1.0 - 2.0 9.90 Gbps 0 78.16 MBytes
2.0 - 3.0 9.90 Gbps 0 78.71 MBytes
3.0 - 4.0 9.90 Gbps 0 78.71 MBytes
4.0 - 5.0 9.90 Gbps 0 78.71 MBytes
5.0 - 6.0 9.90 Gbps 1 80.10 MBytes
6.0 - 7.0 9.90 Gbps 0 80.35 MBytes
7.0 - 8.0 9.90 Gbps 0 80.35 MBytes
8.0 - 9.0 9.90 Gbps 0 82.59 MBytes
9.0 - 10.0 9.81 Gbps 14 41.30 MBytes
Summary
Interval Throughput Retransmits
0.0 - 10.0 9.66 Gbps 21
No further runs scheduled.