Design Details

CHAPI: GPO Implementation of Federation API

CHAPI (for "Clearinghouse API") is the GPO implementation of the Federation API. It includes implementations of the Registry, Slice Authority and Member Authority Federation API's plus some additional methods and services to support the GPO management of accounts and resources.

This document describes the architecture, object models, implementation, configuration and usage details of the CHAPI services.

Architecture

CHAPI was designed as a replacement for the existing GENI prototype Clearinghouse (proto-ch) and is therefore designed to operate against the existing proto-ch database schemas.

CHAPI is built on the AMSoil framework. All CHAPI services are implemented as methods on XMLRPC handlers registered in AMSoil plugins. For more information about AMSoil, see https://github.com/fp7-ofelia/AMsoil

CHAPI methods support a design pattern of:

Handler: Establish the essential endpoint for XMLRPC
Guard: Validate arguments and authorization for method call prior for invoking the implementation
Delegate: Implement the support for the method

The design of Guards will be described below.

AMSoil is designed to work in two modes: debug mode (standalone Python process) and deployed (behind Apache). In the latter mode, a separate python process is spawned to handle each connection, whereas in the former, the requests are handled in series.

The Federation API has two classes of services: unprotected (the Registry) and protected (everything else). Both require an SSL cert and key to make the connection to the XMLRPC endpoint. In the case of unprotected services, the certificate is not validated and no authentication is done. In the case of protected services, the cert is validated that it is signed by one of the trust roots installed for CHAPI and the connection is rejected if not. Note that this authentication step happens at SSL connection set up and AMSoil and CHAPI methods are unaware of any connection failures.

Data Model and Mapping

As noted above, CHAPI services are a backwards-compatible replacement for existing GPO Clearinghouse services. As a resul, CHAPI services work against the existing GPO Clearinghouse database of members, slices, projects etc.

We thus have two different data models:

The 'GPO' model that is represented in the database and the GENI Portal speaks natively
The 'Federation' (i.e. CHAPI) model

CHAPI's service implementations, therefore, must translate between the GPO model as it moves data in and out of the database, and the CHAPI model as it receives requests and returns responses. Furthermore, the GENI portal relies on a particular Clearinghouse API that is not compatible with the CHAPI API. To minimize the risk on Portal development, the portal interface routings (a set of files called sr_client.php, sa_ciient.php, pa_client.php, ma_ciient.php) have been re-written for the CHAPI effort to talk the original Clearinghouse API to portal clients and to translate those between those requests/responses and CHAPI requests/responses.

Among the fundamental differences between the CHAPI and GPO CH API's is the primary keys used to identity objects. In CHAPI (per the Federation API), the primary key for identifying projects, members, slices, slivers and other objects is the URN. In the GPO CH, the primary key is the UID. Much of the work done to translate between the GPO database and Portal queries and the CHAPI services requires translation between UID's and URN's. This is done in the CHAPI service plugins by adding additional queries or query clauses to extract URN's for UID's or vice versa. It is important to note that UID's are unique (at least for a given server) whereas URN's are unique only at a given time: specifically, Slice URN's may be re-used once a slice is renewed and thus a request for a slice UID for a slice URN is not unique. By convention, the mapping of slice URN to slice UID returns the most recent slice UID if there is more than one slice by that URN.

This mulit-layered translation is illustrated in the following diagram.

CHAPI Translation Layers

The tables below describe the set of fields (Federation standard plus CHAPI additional) supported by CHAPI interfaces. In some cases, the fields are derived from particular database tables. In other cases, they are computed. Any fields with prefix "GENI" are non-standard (i.e. CHAPI-specific, not standard to the Federation API) and are advertised in the respective get_verision methods.

Service Registry Data Model

Object	Field	Source
SERVICE	SERVICE_ID	service_registry.id
SERVICE	SERVICE_URN	service_registry.service_urn
SERVICE	SERVICE_URL	service_registry.service_url
SERVICE	SERVICE_CERTIFICATE	service_registry.service_cert
SERVICE	SERVICE_NAME	service_registry.service_name
SERVICE	SERVICE_DESCRIPTION	service_registry.service_description
SERVICE	SERVICE_TYPE	service_registry.service_type

Slice Authority Data Model

Object	Field	Source
SLICE	SLICE_URN	sa_slice.slice_urn
SLICE	SLICE_UID	sa_slice.slice_id
SLICE	SLICE_NAME	sa_slice.slice_name
SLICE	SLICE_DESCRIPTION	sa_slice.slice_description
SLICE	SLICE_EXPIRATION	sa_slice.expiration
SLICE	SLICE_EXPIRED	sa_slice.expired
SLICE	SLICE_CREATION	sa_slice.creation
SLICE	SLICE_PROJECT_URN	computed from project_name
SLICE	_GENI_SLICE_EMAIL	sa_slice.slice_email
SLICE	_GENI_SLICE_OWNER	sa_slice.owner_id
SLICE	_GENI_PROJECT_UID	sa_slce.project_id

PROJECT	PROJECT_URN	computed from project_name
PROJECT	PROJECT_UID	pa_project.project_id
PROJECT	PROJECT_NAME	pa_project.project_name
PROJECT	PROJECT_DESCRIPTION	pa_project.project_purpose
PROJECT	PROJECT_EXPIRATION	pa_project.expiration
PROJECT	PROJECT_EXPIRED	pa_project.expired
PROJECT	PROJECT_CREATION	pa_project.creation
PROJECT	_GENI_PROJECT_EMAIL	pa_project.project_email
PROJECT	_GENI_PROJECT_OWNER	pa_project.lead_id

Member Authority Data Model

Object	Field	Source
MEMBER	MEMBER_URN	ma_member_attribute.name == 'urn'
MEMBER	MEMBER_UID	ma_member.member_id
MEMBER	MEMBER_FIRSTNAME	ma_member_attribute.name=='first_name'
MEMBER	MEMBER_LASTNAME	ma_member_attribute.name=='last_name'
MEMBER	MEMBER_USERNAME	ma_member_attribute,name=='username'
MEMBER	MEMBER_EMAIL	ma_member_attribute.name=='email_address'
MEMBER	_GENI_MEMBER_DISPLAYNAME	ma_member_attribute.name=='displayName'
MEMBER	_GENI_MEMBER_PHONE_NUMBER	ma_member_attribute.name=='telephone_number'
MEMBER	_GENI_MEMBER_AFFILIATION	ma_member_attribute.name=='affiliation'
MEMBER	_GENI_MEMBER_EPPN	ma_member_attribute.name=='eppn'
MEMBER	_GENI_MEMBER_SSL_CERTIFICATE	ma_outside_cert.certificate
MEMBER	_GENI_MEMBER_SSL_PRIVATE_KEY	ma_outside_cert.private_key
MEMBER	_GENI_MEMBER_INSIDE_CERTIFICATE	ma_inside_key.certificate
MEMBER	_GENI_MEMBER_INSIDE_PRIVATE_KEY	ma_inside_key.private_key
MEMBER	_GENI_USER_CREDENTIAL	computed from user certificate
MEMBER	_GENI_CREDENTIALS	computed from user certificate and attributes

KEY	KEY_MEMBER	ma_ssh_key.member_id converted to member_urn
KEY	KEY_ID	ma_ssh_key.id
KEY	KEY_PUBLIC	ma_ssh_key.public_key
KEY	KEY_PRIVATE	ma_ssh_key.private_key
KEY	KEY_DESCRIPTION	ma_ssh_key.description
KEY	_GENI_KEY_MEMBER_UID	ma_ssh_key.member_id
KEY	_GENI_KEY_FILENAME	ma_ssh_key.filename

Services

CHAPI consists of a set of services, each represented by an XMLRPC endpoint and implemented by an AMSoil plugin.

The following table summarizes the services provided by CHAPI:

Service	Description	Federation API	XMLRPC Endpoint
Service Registry	Implementation of Federation API Registry	YES	/SR
Slice Authority	Implementaton of Federation API Slice Authority	YES	/SA
Member Authority	Implementation of Federation API Member Authority	YES	/MA
Logging	Service to support logging events to persistent store	NO	/LOG
PGCH	Implementation of PGCH (ProtoGENI Clearinghouse) API	NO	/PGCH
Credential Store	Supports GENI Portal Authorization	NO	/CS

The sections below describe each service API. For those API's that implement a Federation API service, only those additional (non-standard) API methods will be described.

Service Registry

The Service Registry (SR) implements the Federation Registry API. In addition, it supports the following method:

# Return list of services of given type registered in SR
# The return type is the same as other services (e.g. lookup_aggregates), i.e. a list of tuples with fields of the SERVICE object defined in the Registry API.
# Arguments: service_type (including services not in the standard Federation Registry):
# AGGREGATE_MANAGER = 0;
# SLICE_AUTHORITY = 1;
# PROJECT_AUTHORITY = 2;
# MEMBER_AUTHORITY = 3;
# AUTHORIZATION_SERVICE = 4;
# LOGGING_SERVICE = 5;
# CREDENTIAL_STORE = 6;
# CERTIFICATE_AUTHORITY = 7;
# KEY_MANAGER = 8;
# PGCH = 9;
# WIMAX_SITE = 10;
# IRODS = 11;
# Arguments:
#   service_type : Type of service to match
# Return:
#    List of field/value tuples of SERVICE objects
def get_services_of_type(service_type)

Slice Authority

The Slice Authority (SA) implements the standard Federation Slice Authority API. It supports the SLICE, SLICE_MEMBER, PROJECT, PROJECT_MEMBER and SLIVER_INFO service methods.

In addition, the CHAPI SA supports a set of API calls to support pending requests for joining projects. [NB: This may be moved out of the SA into the GENI Portal after some speaks-for refactoring is completed.].

The following is the set of API calls supported by the SA for managing pending project join requests:

# Create a pending join project request
# Arguments:
#    context_type - type of context (PROJECT==1)
#    context_id - project_id
#    request_type - JOIN_PROJECT = 0, UPDATE_ATTRIBUTES = 1
#    request_text - message of join request
#    request_details - more detailed message to join project
# Return :
#    request_id - identifier of given request
def create_request(self, context_type, \
                       context_id, request_type, request_text, \
                       request_details, credentials, options)

# Resolve a pending join project request (accept, reject, cancel)
# Arguments:
#   context_type - type of context (PROJECT=1)
#   request_id - ID of request to resolve
#   resolution_status - PENDING=0, APPROVED=1, CANCELLED=2, REJECTED=3
#   resolution_description - string describing resolution
#    
def resolve_pending_request(self, context_type, request_id, \
                                resolution_status, resolution_description,  \
                                credentials, options)

# Get all requests for given context (project)
# Arguments:
#   context_type - context of requests (PROJECT=1)
#   context_id - project_id
#   status - status of request (PENDING, APPROVED, CANCELLED, REJECTED)
# Return:
#    List of request field/value dictionaries for requests matching status/context filter
def get_requests_for_context(self, context_type, \
                             context_id, status, \
                             credentials, options)

# Get all requests made by user
# Arguments:
#   member_id - UID of member who created requests
#   context_type - type of context (PROJECT=1)
#   context_id - UID of context (project_id)
# Return:
#   List of name/value dictionaries of request objects
def get_requests_by_user(self, member_id, context_type, \
                             context_id, status, \
                             credentials, options)

# Get all pending requests that can be resolved by given user (lead/admin of project)
# Arguments:
#   member_id - UID of member for whom pending requests are requests
#   context_type - type of context (PROJECT=1)
#   context_id - ID of context (project_id)
# Return:
#    List of dictionaries of name/value pairs of request objects
def get_pending_requests_for_user(self, member_id, \
                                      context_type, context_id, \
                                      credentials, options)

# Get number of all pending requests that can be resolved by given user (lead/admin of project)
def get_number_of_pending_requests_for_user(self, member_id, \
                                                context_type, context_id, \
                                                credentials, options)

# Get request by identifier
# Arguments:
#    request_id : ID of request to lookup
#    context_type : context_type (PROJECT = 1)
# Return:
#  Request structure if context_type of request_id matches context_type argument, None otherwise
def get_request_by_id(self, request_id, context_type, credentials, options)

Member Authority

The Member Authority(MA) implements the standard Federation Member Authority API. It supports the MEMBER and KEY service method sets. In addition, the CHAPI MA supports these additional API calls:

# Create a new member in the Member Authority, based on a dictionary
# of name/value pairs in the attributes arguments including:
#   email_address (e.g. [email protected]) 
#   first_name (e.g. Joe Smith)
#   last_name (e.g. Smith)
#   affiliation (e.g. [email protected];[email protected])
#   eppn (e.g. [email protected]) [Unique]
#   displayName (e.g. Joe Smith)
#   username (e.g. jsmith) [Unique, alphanumeric only]
# Arguments: 
#   attributes - list of name/value/self-asserted tuples of member attributes
#      NB: email_address is required
# Return: 
#    Dictionary of all name/value field pairs for newly created member
def create_member(self, attributes, credentials, options)

Methods for managing user certificates:

# Create a certificate for given user
# Arguments:
#   member_urn - URN of member for whom to create certificate
#   options : 
#       'csr' if a CSR is provided, otherwise create cert and key
# Return:
#    certificate of member signed by MA containing URN, UID, Email information
def create_certificate(self, member_urn, credentials, options)

Methods to manage user authorization of tools (NB: this API will be replaced by speaks-for capability):

# List all client tools known to the MA
# Arguments: None
# Return: List of URN's of client tools known to MA
def list_clients():

# List all client tools member has authorized
# Arguments:
#   member_id
# Return:
#   list of URN's of authorized client tools
def list_authorized_clients(member_id):

# Authorize/Deauthorize client - making a tool able to speak for given user
# Arguments:
#   member_id - UID of member for whom to authorize/deauthorize client
#   client_urn - URN of client tool to authorize/deauthorize
#   authorize_sense - Whether to authorize (True) or deauthorize (False)
def authorize_client(member_id, client_urn, authorize_sense):

Methods to disable/enable members

# Enable/Disable user, making then able/unable to access Federation resources
# Arguments:
#    member_urn - URN of member to enable/disable
#    enable_sense - Whether to enable (True) or disable (False) given member
def enable_user(member_urn, enable_sense, credentials, options):

Methods to add/remove member privileges

# Add a privilege to given user
# Arguments:
#  member_uid - UID of member for whom to add privilege
#  privilege - name of privilege (e.g. 'operator')
def add_member_privilege(member_uid, privilege, credentials, options):

# Revoke privilege from given member
# Arguments:
#   member_uid - UID of member for whom to revoke privilege
#   privilege - name of privilege to revoke
def revoke_member_privilege(member_uid, privilege, credentials, options):

Methods to add/remove member attributes:

# Add attribute of given name/value to given user
# Arguments:
#  member_urn - urn of member to add attribute
#  name - name of attribute
#  value - value of attribute
#  self_asserted - whether the attribute is self-asserted (by member)
def add_member_attribute(member_urn, name, value, self_asserted, credentials, options):

# Remove attribute of given name from given user
# Arguments: 
#   member_urn - urn of member to remove
#   name - name of attribute to remove
def remove_member_attribute(member_urn, name, credentials, options):

Logging

The Logging service supports creating and querying for entries marking particular events in the life of a member's work on a slice or project suitable for display on a user interface.

The following methods constitute the Logging service API:

# Enter new logging entry in database for given sets of attributes                                       
# And logging user (author)      
# Arguments:
#   message - text message of event
#   attributes - dictionary of context_type, context_id pairs 
#  user_id - UUID of member writing the event                                                                       
def log_event(self, message, attributes, user_id)

# Get all entries written by given author in most recent hours     
# Arguments:
#   user_id - UUID of member for whom to query for logged events
#   num_hours - maximum age of log entries to return (default = 24)
# Return:
#   List of log event dictionary of name/value pairs                                      
def get_log_entries_by_author(self, user_id, num_hours)

# Get all entries written for context type/id in most recent hours 
# Arguments:
#   context_type - Type of context associated by attribute (PROJECT=1, SLICE=2)
#   context_id - UUID of object associated by attribute (project_uuid or slice_uuid)
#   num_hours - maximum age of log entries to return (default = 24)
# Return:
#   List of log event dictionary of name/value pairs                                      
def get_log_entries_for_context(self, context_type, context_id, num_hours)

# Get all log entries corresponding to the UNION of a set                                                
# of context/id pairs in most recent hours    
# Arguments:
#   attribute_sets : List of context_id/context_type dictionary pairs
#   num_hours
# Return:
#   List of log event dictionary of name/value pairs                                                                                               
def get_log_entries_by_attributes(self, attribute_sets, num_hours)

# Get set of attributes for given log entry
# Arguments:
#   event_id : ID key for logged event
# Return:
#    Dictionary of name/value pairs context_type/context_id                                                              
def get_attributes_for_log_entry(self, event_id)

PGCH

The PGCH service implements a subset the PGCH (ProtoGENI Clearinghouse) API, documented in http://www.protogeni.net/ProtoGeni/wiki/ClearingHouseAPI1. The following table describes the set of PGCH calls supported by the PGCH service at the /PGCH XMLRPC endpoint.

Method	Arguments	Description
GetVersion	None	Get version and API details about PG Clearinghouse
GetCredential	'args' : None (user credential) or type=slice, urn=slice_urn/uuid=slice_uid (slice credential)	Generate a user credential or slice credential
Resolve	'args' : credential, hrn, urn, uuid, type (slice or user)	Lookup given slice based on credential, hrn or uuid, return dictionary of uuid, creator_uuid, creator_urn, slice_uid
Register	'args' : credential, hrn, urn, type=slice, 'cred' : slice credential	Register slice with slice authority
RenewSlice	'args' : credential, expiration	Renew slice to given expiration, return new slice credential
GetKeys	'args' : credential	Return SSH public keys associated with given user associated with user credential
ListComponents	'args' : user credential	Return list of all aggregate (component) managers : Dictionary of 'gid' 'hrn' 'url' for each AM

Credential Store

The Credential Store (CS) service supports GENI Portal authorization, providing a set of attributes and privileges that a given user posesses ona context-free or context (project, slice) basis. The following are the API calls supported by the CS Service:

# Get attributes of given principal (a member, by UID) in a given context (project/slice by UID)
# Arguments:
#    principal: member_uid for whom to gather attributes
#    context : ID of context (slice_uid, project_uid)
# Return:
#    List of attributes (context, role) for projects, slices and general (non-contextual) attributes
def get_attributes(principal, context_type, context, credentials, options)

# Get permissions (action, context_type, context_id tuples) for a given principal (member by UID)
# Arguments:
#   principal: UID of member for who to gather permissions
# Return:
#   List of permission name, context_type (PROJECT=1, SLICE=2), context_id (slice_id, project_id) for which member has particular privileges
def get_permissions(principal, credentials, options)

Guard-based Validation and Authorization

CHAPI methods are invoked via XMLRPC/SSL. Calls that are protected and not authenticated are immediately rejected. From there, the request goes through a set of steps:

The invocation is checked for speaks-for credentials, and if so, the 'true' caller (the one being spoken for) replaces the evident caller (the tool doing the speaking)
The Guard for that method is invoked to check that the call is valid and authorized
The method is handed to the Delegate where it invoked and result is returned
If any exception is encountered in method invocation, it is trapped and its message and code are sent back in the standard [code, value, output] tuple.

This section describes the logic underlying the Guard.

The guard performs two essential functions, which will be discussed in subsections: 1) Arguemnt checking and 2) Authorization.

Guard Argument Checking

Guards are provided with a set of per-method rules for checking invocation arguments. Different methods have different sets of required options, and permitted options, and within those options (espeically 'field' and 'filter' and 'match' options) different required and permitted fields. The Guards check the invocation against these required and permitted rules and raise exceptions accordingly.

The Guards also consult the tables of argument, option or field types and make sure the values are well formatted (e.g. a DATETIME is YYYY-MM-DD HH:MM:SS). If not, the Guard raises exceptions accordingly.

Guard Authorization

The Guards apply the ABAC logic prover to determine if a given invoking principal has the authorization to invoke a particular method with a particular set of arguments. For this computaton, we need several pieces:

The method must specify a set of policies that indicate what assertions must be true about the caller for the caller to be allowed to make the call
The method must specify a set of assertion constructors which will generate relevant assertions about the caller

With the set of policies and the set of assertions, the query asking "is this user allowed to call this method in this context?" is computed. If the query cannot be prove, an authorization exception is raised.

The following is an example of how the Guard Authorization works:

Here is a statement from the SA Guard indicating the authorization behaviors around invoking the 'get_credentials' method:

        'get_credentials' : \
            SubjectInvocationCheck([
                "ME.MAY_GET_CREDENTIALS<-ME.IS_OPERATOR",
                "ME.MAY_GET_CREDENTIALS_$SUBJECT<-ME.BELONGS_TO_$SUBJECT",
                ], assert_belongs_to_slice, slice_urn_extractor),

What this means is:

We extract the context from the 'slice_urn' argument of the call
We generate assertions based on the slices to which the caller belongs.
Our policy for who may call get_credentials is:
An operator (someone with the 'operator' attribute)
Someone who is a member of the slice

When a caller invokes get_credentials with slice_urn urn:publicid:IDN+ch-mb.gpolab.bbn.com:PRESIDENT+slice+LINCOLN,

We generate assertions based on slice membership. Specifically, for each slice X to which the caller belongs, the assertion ME.BELONGS_TO_X<-CALLER is generated where "ME" is the SA and "CALLER" is the invoker.
We generate standard assertions about the caller based on their attributes
We generate "ME.IS_$CALLER<-CALLER", i.e. for user urn urn:publicid:IDN+ch-mb.gpolab.bbn.com+user+jsmith, we flatten the user URN and generate ME.IS_urn_publicid_IDN_ch_mb_gpolab_bbn_com_user_jsmith<-CALLER
If the caller is an operator, we generate ME.IS_OPERATOR<-CALLER
If the caller is a project_lead, we generate ME.IS_PI<-CALLER
If the caller is an authority, we generate ME.IS_AUTHORITY<-CALLER
We instantiate the templated policies, in this case, we generate these statements, replacing '$SUBJECT with the flattened slice URN: urn_publicid_IDN_ch_mb_gpolab_bbn_com_PRESIDENT_slice_LINCOLN
ME.MAY_GET_CREDENTIALS<-ME.IS_OPERATOR
ME.MAY_GET_CREDENTIALS_urn_publicid_IDN_ch_mb_gpolab_bbn_com_PRESIDENT_slice_LINCOLN<-ME.BELONGS_TO_urn_publicid_IDN_ch_mb_gpolab_bbn_com_PRESIDENT_slice_LINCOLN
For methods involving operations on specific slices, we generate assertions describing the membership relationship (if any) between the caller and that slice, e.g. ME.IS_LEAD_$slice_urn<-CALLER.
For methods involving operations on specific projects, we generate assertions describing the membership relationship (if any) between the caller and that project, e.g. ME.IS_ADMIN_$project_urn<-CALLER.
For methods involving operations on specific members, we generate assertions describing the relationship (if any) between the caller and that member e.g. ME.SHARES_SLICE_$member_urn<-CALLER.

Given this set of assertions annd poliies, we then try to prove either of the statements

ME.MAY_GET_CREDENTIALS<-CALLER
ME.MAY_GET_CREDENTIALS_urn_publicid_IDN_ch_mb_gpolab_bbn_com_PRESIDENT_slice_LINCOLN<-CALLER

Note that proving a such a disjunction requires two separate ABAC proofs.

CHAPI Tools

Two command-line tools are available for using, testing and debugging CHAPI.

client.py

client.py is a python program (in chapi/tools) that constructs and sewnds properly formed Federation API requests to a service URL and receives and prints the result.

client.py takes "--method <method_name>" and "--url <service_url>" arguments to specify the API method name and URL of service provider.

In addition, because many Federation API calls take a list of credentials and a dictionary of options, client.py supports reading these arguments from files:

options: from a JSON file with the argument --options_file
credentials: wiht a list of comma-separated credential filenames

Beyond that, all Federation API calls consist of sets of arguemnts of type URN, UID, STRING or INT which can be provided on the command line.

For example, to renew a slice, the following command will renew slice FORD in project VEEP

python client.py --method update_slice --url https://localhost/SA \
       --options_file renew.json --urn urn:publicid:IDN+ch-mb.gpolab.bbn.com:VEEP+slice+FORD

provided the following renew.json file

{
        "fields" : {"SLICE_EXPIRATION" : "2013-12-01 12:00:00"}
}

The following is the '--help' documentation from the program:

Usage: client.py [options]

Options:
  -h, --help            show this help message and exit
  --url=URL             Server URL
  --urn=URN             URN for API arguments
  --key=KEY             Location of user key
  --cert=CERT           Location of user cert
  --method=METHOD       Name of method to invoke
  --agg_url=AGG_URL     URL of aggregate in some API calls
  --string_arg=STRING_ARG
                        String argument for some calls
  --string2_arg=STRING2_ARG
                        second string argument for some calls
  --int_arg=INT_ARG     Integer argument for some calls
  --int2_arg=INT2_ARG   second integer argument for some calls
  --int3_arg=INT3_ARG   third integer argument for some calls
  --uuid_arg=UUID_ARG   UUID argument for some calls
  --uuid2_arg=UUID2_ARG
                        second UUID argument for some calls
  --uuid3_arg=UUID3_ARG
                        third UUID argument for some calls
  --file_arg=FILE_ARG   FILE argument for some calls
  --options=OPTIONS     JSON of options argument
  --options_file=OPTIONS_FILE
                        File containing JSON of options argument
  --attributes=ATTRIBUTES
                        JSON of attributes argument
  --attributes_file=ATTRIBUTES_FILE
                        File containing JSON of attributes argument
  --credentials=CREDENTIALS
                        List of comma-separated credential files

pgch_client.py

The PGCH ciient speaks the PGCH (ProtoGENI Clearinghouse) protocol to any server speaking the PGCH protocol. It is designed to test the PGCH XMLRPC endpoint on CHAPI.

Like client.py above, pgch_client takes "--method=" and "--url=<service_url" arguments.

PGCH takes an arguemnts dictionary argument so PGCH takes an argument json file specified with the "--args_file=<args_file>" argument.

Example: The following call will extract a slice credetial from a given PG-compatible SA

python pgch_client.py --url https://localhost:8501/PGCH --method GetCredential \
     --args_file /tmp/get_credential_args.json

for the given argument file:

{
        "type" : "slice",
        "urn" : "urn:publicid:IDN+ch-dm.gpolab.bbn.com:PFOOT+slice+PBAR"
}

The following is the '--help' documentation from the pgch_client.py program:

Usage: pgch_client.py [options]

Options:
  -h, --help            show this help message and exit
  --url=URL             Server URL
  --key=KEY             Location of user key
  --cert=CERT           Location of user cert
  --method=METHOD       Name of method to invoke
  --args=ARGS           JSON of args argument
  --args_file=ARGS_FILE
                        File containing JSON of args argument

CHAPI Code Utilities and Structure

Directory structure

CHAPI code is divided into directories per plugin:

plugins/chapiv1rpc - the base classes for delegate, handler, guard for the Federation API
plugins/sarm - the plugin and implementation of the SA delegate and SA guard
plugins/marm - the plugin and implementation of the MA delegate and MA guard
plugins/logging - definition of the CHAPI logging service
plugins/pgch - definition of the PGCH delegate and guard
plugins/srm - definition of the CS delegation and guard

Each of these plugin directories are installed into the AMSoil installation by placing a symbolic link into the AMSoil plugins directory (i.e. AMSoil/plugins/sarm => CHAPI/plugins/sarm, etc.)

Additionally, there is a directory called 'tools' (parallel to plugins) that contains tools and utilities for building plugin capabilities as well as running/testing CHAPI capabilities.

Code Utilities

chapi_log.py

In the tools module, there is a package called chapi_log.py. This package provides methods to route logging messages from CHAPI to different destimations (message files, log files, std err, etc.). The CHAPI code calls the routines in this package (chapi_log, chapi_warn, chapi_info, chapi_debug, chap_log_invocation, chapi_log_result, chapi_log_exception) and then this package can be configured to set up standard handling of different message types for a given installation.

An additional method called chap_audit allows for logging specific audit-worthy events (e.g. creating users or changing their roles, privileges) in some particular or redundant manner.

Currently the messaging goes to a log file in /tmp/chapi.log. In the future, we expect this logging to be parameterized by values in the Parameters.py file.

data caching

Some data mappings, for example between URN and UID, are relatively static and can be efficiently cached in the CHAPI python process.

The utility guard_utils.py provides many of the routines to translate between CHAPI and GPO formats. Specifically, there is a thread local variable _context that contains an attribute named 'cache'. These variables are managed by routines called cache_get (get a particular named portion of the data cache) and cache_clear (clear the entire data cache) This cache is a dictionary that holds mappings of variables with constant relationships. The following is an example of the code and how it is used:

cache = cache_get('slice_uid_to_urn')
if slice_uid in cache:
   slice_urn = cache[slice_uid]
else:
   slice_urn = .... # Lookup urn from uid in DB
   cache[slice_uid]=slice_urn

This cache mechanism also supports an interface called 'memoize' that automatically saves/restores values from wrapped calls in a cache tailored for that call. For example,

# *** This code maintains a cache of project names from slice URN and consults the cache before invoking the code below
@memoize
# Get the project name from a slice URN                                                                  
def lookup_project_name_for_slice(slice_urn):
    parts = slice_urn.split("+")
    authority = parts[1]
    authority_parts = authority.split(":")
    project_name = authority_parts[1]
    return project_name

CHAPI Parameters and Customization

The CHAPI services are configured by a parameters file Parameters.py in chapi/chapi/chapiv1rpc/chapi.

The file contains a variable 'parameters' that has a list of 'name/val/desc' dictionary tuples. The file is parsed at AMSoil startup and the values are placed in the AMSoil config service; thus, these parameters are available from a config.get call.

The following table summarizes the parameters that configure the CHAPI services in Parameters.py:

Parameter Name	Parameter Description
chapiv1rpc.ch_cert_root	Folder which includes trusted clearinghouse certificates for GENI API v3 (in .pem format). If relative path, the root is assumed to be git repo root.
chapiv1rpc.ch_cert	Location of CH certificate
chapiv1rpc.ch_key	Location of CH private key
chapi.ma_cert	Location of MA certificate
chapi.ma_key	Location of MA private key
chapi.sa_cert	Location of SA certificate
chapi.sa_key	Location of SA private key
chapi.log_file	Location of chapi's log file
chrm.authority	name of CH/SA/MA authority
flask.debug.client_cert_file	Debug client cert file
chrm.db_url	CHAPI database URL
flask.fcgi	Use FCGI server instead of the development server.
flask.fcgi_port	Port to bind the Flask RPC to (FCGI server).
flask.app_port	Port to bind the Flask RPC to (standalone server).
flask.debug	Write logging messages for the Flask RPC server.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly