Networking, Lens API Quick Start Guide

Required access credentials

Lens API clients must use an IP address that is registered in DNS with a hostname ending in uiuc.edu, illinois.edu or uillinois.edu.  Access to the Lens API requires a current campus Active Directory user account whose name is known as a network contact in Technology Services Contacts Database. The Lens API provides users with data pertaining to the networks for which they are listed contacts.

Access by Net ID

Your Net ID is represented by an [AD.UILLINOIS.EDU] Active Directory user account which identifies you personally. As a network contact, you can access the Lens API using your Net ID and AD password. This is appropriate for your direct, interactive use of the API because you can supply your name and password as necessary. For example, you can use your Net ID with command-line utilities you create and for direct browser-based access.

Other applications require Lens API access when no user is involved. For these we do not recommend using your personal Net ID and AD password when such use requires you to place them in persistent storage (e.g. a disk file).

Storing your password on disk risks more than unauthorized Lens API access. At potential risk are any other accessible University resources with which you are entrusted.

See the following section for using the Lens API with unattended applications.

Access by pseudouser

Some applications use the Lens API when no user is involved who can provide access credentials (e.g. scheduled jobs). For these you can use a campus AD user account you create within your own organizational unit and dedicate to this purpose. This pseudouser account represents your application and neither you nor any other individual person. You can store the pseudouser password for your unattended application without risk to personal and unrelated resources.

We strongly recommend you choose a username that begins with your unit
 or group followed by the hyphen character in order to identify the
responsible unit and sidestep potential conflicts with Net IDs. (For
several years, new Net IDs have been created to exclude the hyphen
character.) For example, pseudousers for the Physics or Computer Science
departments might be named "physics-lens-user" or "compsci-lens-user".

As for any Net ID, the Lens API imparts privileges to your pseudouser account matching its Technology ServicesContacts Database network contact assignments. Unlike the Net IDs, however, Contacts Database does not automatically know that your pseudouser exists. You need to contact net-trouble@illinois.edu to request a Contacts Database identity made with the same name as your non-Net ID pseudouser account. Afterward, you or an authorized network contact in your unit can assign your pseudouser to be read-only contacts for desired networks.

We strongly recommend you assign read-only privileges! Although today the Lens API just provides read-only service, the pseudouser is also known to and carries privileges within Iris! Furthermore, the Lens API may not always remain read-only.

Through pseudouser accounts, you can automate your applications with little need for contact beyond your unit. Although they mitigate risk, they nonetheless access University resources and can be exploited to compromise privacy. When you create and empower pseudouser accounts as agents on your behalf or your unit's, you retain responsibility for its proper use and protection. In short: you made it, you own it, you authorized it, and you are responsible for it.

General query format

The Lens API defines several types of objects which may be returned by HTTP GET queries. (The POST method is not supported at this time. See the Lens API FAQ.)

All objects of like type provide the same set of attributes. For example, device objects all have a device_name attribute. These attributes may be used in queries to retrieve objects, and they are returned by queries as a part of the objects. Furthermore, objects are related to each other. For example, device objects are related to interface objects (because network devices have interfaces). The goal of every Lens API query is to fetch a list of some particular type of object based on the values of attributes and their interrelationships. The list may be "dressed up" with additional information to build a consistent and informative data set in one query.

All Lens API queries have a format described by the following template:

https://lens-api.cites.illinois.edu/lens/uiuc-lens/<subject type>?
	 <param1>=<expression1>&<param2>=<expression2>&...&dressings=<dressing1>,<dressing2>,...

The subject type is the type of the objects you want to list (e.g. device). Each parameter can be the name of an attribute that is found in the subject type (e.g. param1 may be device_name). An expression describes the attribute values you're looking to match in choosing objects for your list. Alternatively, a parameter can describe how the subjects in your list must relate to other objects. The list of objects reflects the application of all these qualifications as described below. Finally, the result is elaborated with additional data as requested by the given set of "dressing" tokens (e.g. dressing1).

Example: device

To retrieve Lens device object data, point your browser to

https://lens-api.cites.illinois.edu/lens/uiuc-lens/device 

You will be prompted for your netID and AD password in ad-dc-t1.ad the first time you submit a request. The URL above retrieves all devices stored in the Lens database that include vlans of your privileged network. The device attributes returned will be the "essential" attributes listed in the device specification.

Supported object types

Here is a list of all currently supported object types, linking to their specifications. Consult each specification to understand what the type represents, the attributes it supports, and how its objects relate to others.

• device
• device_vlan
• interface
• interface_vlan
• ip_mac
• mac_port
• password
• subnet
• vlan

Query mechanics

Object attribute data types

Objects such as devices can also be retrieved using any of their attributes as search criteria. The attribute names are listed in the specification, as are their basic data types. An attribute's data type defines the formats for its query expressions and its values as returned by queries. The data types currently supported are:

Regular Expressions

The webservices supports POSIX Extended Regular Expressions for all text data types. To invoke a regular expression match, the string must begin with the tilde "~" character. The API forces all regular expression matching to be case-insensitive. The POSIX Extended Regular Expressions or ERE flavor is a standard similar to the one used by the UNIX egrep command. For a more complete description of POSIX ERE regular expressions, see POSIX Extended Regular Expressions. For a table comparing various regular expression flavors, see Regular-Expressions.info.

Range queries

The webservices recognizes the comma "," as a range separator for all numeric, date, or scalar IP data types. If a comma is used, the webservice will perform a fully-inclusive range match. For example, using "vlan_numer=100" will result in an exact match for vlan_number 100, while using "vlan_number=100,200" will result in a range scan for all vlan_numbers between 100 and 200, inclusive. IP datatypes allow single IP range scans, such as "192.168.100.0,192.180.100.255", but do not allow CIDR-notation mixed with the comma range separator ("192.168.100.0/24,192.169.101.255") as the desired outcome is unclear.

Semantics of joining multiple parameter queries

In all cases, different parameters will be "AND"ed, while multiples of the same parameter will be "OR"d. So "device_name=sw-dcl1&type=switch" translates to "device_name=sw-dcl1 AND type=switch" while "device_name=sw-dcl1&type=switch&device_name=sw-dcl5" translates to "(device_name=sw-dcl1 OR device_name=sw-dcl5) AND type=switch".

Relationship-based queries

All object types have relationships defined in their specification. Each relationship has a name and represents a real-world semantic link between an object and others. For example, a device object relates to several interface objects via its interfaces relationship. A relationship has only one "related type". Objects can be queried using relationships and the attributes of related types by the following syntax:

<{{subject_type}}>?<{{relationship_name}}>:<{{related_object_attribute}}>=<expression> 

A query by relationship lists the objects (of subject_type) which relate (via relationship relationship_name) to a selection of objects (of the related type) matching the attribute query expression. Note that you may build your queries with simultaneous qualifications by attributes and by relationship, subject to the multiple-parameter semantics described in the previous section.

For example, to find all devices the have an interface that is named "A1", you would use the following query:

device?interfaces:ifname=A1 

device is the base object, interfaces is a device relationship that references interface objects, and ifname is an attribute of an interface.

Note that relationships function similarily to attributes in the syntax above, making it possible to follow a "chain" of relationships to a distantly related object. For example, one could extend the previous example to find all devices that have an interface that is configured for vlan 32 untagged by using this query:

device?untagged_interfaces:vlans:vlan_number=32

The device object's "untagged_interfaces" relationship points to interface objects, which themselves have a relationship called "vlans" which points to interface_vlan objects that have the attribute "vlan_number." This query will return all device objects that include in their set of "untagged_interfaces" an interface that has within its set of "vlans" an interface_vlan object with vlan_number=32.

Object dressings

A basic query will only return a list of objects of the subject type that satisfies the query parameters. In order to get related objects returned in the same query, you must apply "dressings" to the result. Each Relationship defined for an object in the specification provides a "Dressing Parameter" to invoke the dressing. Dressings are requested by including the "dressings" parameter followed by a comma-separated list of dressings in the order you wish them applied to your result. For example, to request devices with device_name=sw-dcl5 and all of that device's interfaces:

device?device_name=sw-dcl5&dressings=dev_interfaces 

The objects returned in a dressing call can also be dressed. To request the same device with all its interfaces and those interfaces' vlans:

device?device_name=sw-dcl5&dressings=dev_interfaces,ifc_vlans 

Note that the ordering of the dressings listed in the request is important. When the "dev_interfaces" dressing is applied, interface objects are added to the result set. The "ifc_vlans" dressing is then applied to those interface objects. If "ifc_vlans" is requested first, there would be no interface objects in the result set to modify, so that dressing would be empty. Dressings are always applied to every object of the dressing's target type in the result set, no matter where in the list that object was applied.

How the API returns data

Result structure

See the following link to learn about the data structures returned by the Lens API. Note: This page is taken from early design stages. While the result structure is accurately defined, please note that the examples do not reflect the true names of object types and attributes. You'll still get the idea.

Lens API - Result Object

Serialized data encodings

In order to package an interrelated and potentially large data set, the Lens API returns a complex data structure. The data structure is serialized in one of the available encodings:

Selecting an encoding

You can select an encoding by the following means:

• Include a Content-Type header in your HTTP request with a value indicating the MIME type of the desired encoding. For example:

Content-Type: application/json

device?device_name=~sw.*&content-type=text/xml 

• Supply a content-type URL query parameter with a value indicating the MIME type of the desired encoding. For example:

More examples

The root url for all examples is: https://lens-api.cites.illinois.edu/lens/uiuc-lens/

device?type=switch&opsys=hp&opsys=os 

The above query returns devices whose type="switch" and (opsys="hp" or opsys="os").

device?type=~^Swi&opsys=~^hp&opsys=os 

The above query returns devices whose type satisfies the regular expression /^Swi/ and (opsys satisfies /^hp/ or opsys="os"), all case insensitive. Regular expressions can be used for string columns such as name, type, created_by, model, opsys, by prefixing the string with the tilde "~".

device?ip=172.20.14.47 

The above query returns devices with an IP Address of 172.20.14.47

device?ip=172.20.14.0/23 

The above query returns devices with an IP Address in the range of 172.20.14.0 through 172.20.15.255

mac_port?mac=001122334455&mac=aabb1122ccff 

The above query returns a list of current ip_mac observations where the MAC address matches either 001122334455 or aabb1122ccff

ip_mac?ip=172.20.14.47&ip=172.20.14.48 

The above query returns a list of ip_mac objects whose IPs match either 172.20.14.47 or 172.20.14.48.

Scripting

In a scripting language, such as perl, you can specify the return mime type as well as your credentials to allow httpd basic authentication over SSL.

#!/usr/bin/perl -w
use strict;
my $url='https://lens-api.cites.illinois.edu/lens/uiuc-lens/device?device_name=sw-dcl5';

my @results = get($url);
foreach my $result (@results) {
  print $result."\n";
}

sub get {
  use LWP::UserAgent;
  my $addr = shift;
  my $req = HTTP::Request->new(GET => $addr, HTTP::Headers->new);
  $req->header(Accept => 'text/xml');  # substitute your favorite mime type here
  my $auth = "Basic ".ncode("mynetid:mysupersecretpassword");  # netID and ad password
  $req->header(Authorization => $auth);
  my $res = LWP::UserAgent->new->request($req);
  die $res->status_line unless $res->is_success;
  return $res->content;
}
sub ncode {
  use MIME::Base64;
  my $in = shift;
  return encode_base64($in);
}
1;

The result displays in the standard output. Currently, text/xml, text/x-data-dumper, text/x-json, text/x-yaml mime types are supported.