Skip to content

Commit

Permalink
Gsa/merge main into dev (#380)
Browse files Browse the repository at this point in the history
* Gsa/staging/production release (#148)

* Bump get-func-name from 2.0.0 to 2.0.2 in /front-end

Bumps [get-func-name](https://github.com/chaijs/get-func-name) from 2.0.0 to 2.0.2.
- [Release notes](https://github.com/chaijs/get-func-name/releases)
- [Commits](https://github.com/chaijs/get-func-name/commits/v2.0.2)

---
updated-dependencies:
- dependency-name: get-func-name
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump @vue/eslint-config-prettier from 7.1.0 to 8.0.0 in /front-end

Bumps [@vue/eslint-config-prettier](https://github.com/vuejs/eslint-config-prettier) from 7.1.0 to 8.0.0.
- [Release notes](https://github.com/vuejs/eslint-config-prettier/releases)
- [Changelog](https://github.com/vuejs/eslint-config-prettier/blob/main/CHANGELOG.md)
- [Commits](vuejs/eslint-config-prettier@v7.1.0...v8.0.0)

---
updated-dependencies:
- dependency-name: "@vue/eslint-config-prettier"
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump eslint-plugin-vue from 9.16.1 to 9.17.0 in /front-end

Bumps [eslint-plugin-vue](https://github.com/vuejs/eslint-plugin-vue) from 9.16.1 to 9.17.0.
- [Release notes](https://github.com/vuejs/eslint-plugin-vue/releases)
- [Commits](vuejs/eslint-plugin-vue@v9.16.1...v9.17.0)

---
updated-dependencies:
- dependency-name: eslint-plugin-vue
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump wheel from 0.41.1 to 0.41.2

Bumps [wheel](https://github.com/pypa/wheel) from 0.41.1 to 0.41.2.
- [Changelog](https://github.com/pypa/wheel/blob/main/docs/news.rst)
- [Commits](pypa/wheel@0.41.1...0.41.2)

---
updated-dependencies:
- dependency-name: wheel
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump @rushstack/eslint-patch from 1.3.2 to 1.5.1 in /front-end

Bumps [@rushstack/eslint-patch](https://github.com/microsoft/rushstack/tree/HEAD/eslint/eslint-patch) from 1.3.2 to 1.5.1.
- [Changelog](https://github.com/microsoft/rushstack/blob/main/eslint/eslint-patch/CHANGELOG.md)
- [Commits](https://github.com/microsoft/rushstack/commits/@rushstack/eslint-patch_v1.5.1/eslint/eslint-patch)

---
updated-dependencies:
- dependency-name: "@rushstack/eslint-patch"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump @cypress/request and cypress in /front-end

Bumps [@cypress/request](https://github.com/cypress-io/request) to 3.0.1 and updates ancestor dependency [cypress](https://github.com/cypress-io/cypress). These dependencies need to be updated together.


Updates `@cypress/request` from 2.88.12 to 3.0.1
- [Release notes](https://github.com/cypress-io/request/releases)
- [Changelog](https://github.com/cypress-io/request/blob/master/CHANGELOG.md)
- [Commits](cypress-io/request@v2.88.12...v3.0.1)

Updates `cypress` from 12.17.3 to 13.2.0
- [Release notes](https://github.com/cypress-io/cypress/releases)
- [Changelog](https://github.com/cypress-io/cypress/blob/develop/CHANGELOG.md)
- [Commits](cypress-io/cypress@v12.17.3...v13.2.0)

---
updated-dependencies:
- dependency-name: "@cypress/request"
  dependency-type: indirect
- dependency-name: cypress
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating build and run scripts

In part, the new top-level `npm run dev` command will run both the
backend and the frontend together.

* Updating .nvmrc

* Further updates to package build, run, and test scripts

* Updating README python instructions

Minor errors in fixed

* Updating lint config

`npm run lint` should now work correctly,with some initial settings
for Vue

* Undo linting changes

* Updating eslint config to specify JS files as modules

* Update README.md

* Update README.md

* Bump eslint from 8.47.0 to 8.51.0 in /front-end

Bumps [eslint](https://github.com/eslint/eslint) from 8.47.0 to 8.51.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](eslint/eslint@v8.47.0...v8.51.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* update eslint to fix version conflict with prettier. Fix various linting errors

* add test coverage check

* revert prettier formatting from uswds files

* remove coverage-c8 dependency

* add workflow for front-end tests

* make test for date use UTC explicitly

* make utc explicit in date formating

* use recommended eslint settings from Eric - thank you - and fix additional linter warnings

* merge

* Combining READMEs

* Removing flaky test suite

* Fixing typo

* Adding top-level links to jump straight to instructions

Lets devs jump right to what they probably care about most at first

* Updating SAM key link

* Adding note about e2e tests

* Updated uswds version from 3.6.0 to 3.6.1

* Updated footer based on Issue 123.

Added NASA logo to the project to display in the footer.

* Updated footer css layout for responsiveness

* Updated footer logo placement on mobile.
Fixed footer broken unit test.

* Updated footer format to support mobile.
Override uswds default settings to allow more flex options.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mark Meyer <[email protected]>
Co-authored-by: eric-gade <[email protected]>
Co-authored-by: Eric Gade <[email protected]>
Co-authored-by: Tim Hollosy <[email protected]>

* Gsa/issue 166/alert banner (#168)

* Added alert to home page to notify user possible issues with the search tool.

* Updated alert language based client feedback.

* Updated the alert message for search issue.

* Gsa/release/staging (#171)

* Updated uswds version from 3.6.1 to 3.7.1 (#164)

---------

Co-authored-by: Mark Meyer <[email protected]>
Co-authored-by: eric-gade <[email protected]>
Co-authored-by: Eric Gade <[email protected]>
Co-authored-by: Tim Hollosy <[email protected]>

* Addressed various dependency vulnerabilities in front-end and back-end that were flagged by dependabot. (#189)

* Sprint 25 (#201)

Issue #190 Dependabot Alert: FastAPI Content-Type Header ReDoS

* Production Release (#207)

Sprint 26 Changes:
Issue #200 889 Footer Identifier Update to include Domain

* Production Release (#211)

Includes the following issue(s):
Issue #203 Update USWDS from 3.71 to 3.8 | 889 Tool

* Production Release (#223)

Includes the following issues:

Issue #218 Dependabot Alert: Request smuggling leading to endpoint restriction bypass in Gunicorn

* Production Release (#249)

Sprint 32 Issues:

Update USWDS from 3.8 to 3.8.1 |889 Tool #237
Dependabot Alert: Werkzeug debugger vulnerable to remote execution when interacting with attacker controlled domain #228
Dependabot Alert: follow-redirects' Proxy-Authorization header kept across hosts #208

* Production Release (#259)

Sprint 33 include(s) the following issues:

Improve performance by adding explicit height and width to image elements #146
Interactive elements indicate their purpose and state #185

* Production Release (#277)

Includes Spring 34 and 35 issues.

Implement the Link Checker on the 889 Tool #151
Research Issues found with Lighthouse  #162
Research Issues found with Lighthouse and address if applicable #269
Dependabot Alert: WS Affected by a DoS When Handlin a Request with many HTTP Headers #261

* Spell out FOUO (#284)
remove banner. (#285)

* Update to the latest version of USWDS 3.8.2 | 889 Tool #299
Dependabot Alert: Axios Cross-Site Request Forgery #278

* Production Release (#336)

* Commit includes the following::

Add Expiration Date field to the Search Results Information Displayed on the Results Screen #318
Dependabot Alert: Regular Expression Denial of Service (ReDoS) in micromatch #319

* Update packages.

* Production Release (#352)

Issue #333 Dependabot Alert: path-to-regexp outputs backtracking regular expressions
Issue #332 Dependabot Alert: DOMPurify allows tampering by prototype pollution
Issue #346 Research SAM.gov to determine why vendors/contractors marked to be shared are not appearing in the 889 Tool
Issue #327 Bug: Accessibility Issue; Lists do not contain only <li> elements and script supporting elements ( <script> and <template>)

* Production Release (Sprint 41) (#361)

Dependabot Alert: Vite DOM Clobbering gadget found in vite bundled scripts that leads to XSS #334
Dependabot Alert: Vite's server.fs.deny is bypassed when using ?import&raw #335

* Sprint 42 (#372)

* Merged staging into main (#379)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Mark Meyer <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: eric-gade <[email protected]>
Co-authored-by: Eric Gade <[email protected]>
Co-authored-by: Tim Hollosy <[email protected]>
Co-authored-by: John Labbate <[email protected]>
Co-authored-by: John Labbate <[email protected]>
  • Loading branch information
8 people authored Nov 19, 2024
1 parent 40688d0 commit ae042f2
Show file tree
Hide file tree
Showing 56 changed files with 6,288 additions and 7,208 deletions.
11 changes: 0 additions & 11 deletions .eslintrc.json

This file was deleted.

29 changes: 29 additions & 0 deletions .github/workflows/broken-link-checker.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Broken Link Checker
on:
push:
paths:
- front-end/**
- .github/**

jobs:
broken-link-checker:
runs-on: ubuntu-latest
defaults:
run:
working-directory: 'front-end'
steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Setup Node.js
uses: actions/setup-node@v2
with:
node-version: '18'
cache: 'npm'
cache-dependency-path: front-end/package-lock.json

- name: install dependencies
run: npm ci

- name: Run broken link checker
run: npm run link-checker:ci
1 change: 0 additions & 1 deletion .github/workflows/deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ jobs:
cf_space: ${{ inputs.environment }}
command: >-
bin/cg-set-env.sh ${{ inputs.environment }}
SAM_API_KEY=${{ secrets.SAM_API_KEY }}
- name: Deploy the application
uses: cloud-gov/cg-cli-tools@main
Expand Down
33 changes: 33 additions & 0 deletions .github/workflows/frontend-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Frontend tests and code coverage

on:
push:
paths:
- front-end/**
- .github/**

jobs:
unit-tests-and-lint:
runs-on: ubuntu-latest
defaults:
run:
working-directory: 'front-end'
steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Setup Node.js
uses: actions/setup-node@v2
with:
node-version: '18'
cache: 'npm'
cache-dependency-path: front-end/package-lock.json

- name: install dependencies
run: npm ci

- name: run linter
run: npm run lint

- name: code run coverage > 90 %
run: npm run test:coverage -- --watch=false
1 change: 1 addition & 0 deletions .nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
18
188 changes: 107 additions & 81 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,48 +1,31 @@
# GSA SmartPay 889 Representations SAM.gov Tool

Jump to:
* [Development setup](#development-and-testing)
* [Production deployment](#production-deployment)
* [Features and API description](#features)

## Overview
The SmartPay 889 Representations SAM.gov Tool is composed of a backend API written in python, and a web-based frontend.

This search uses the openGSA sam.gov Entity Management API. Vendors that have selected "DOES NOT" for both FAR 52.204-26(c)(1) and (2) are marked as compliant. A vendor is not selectable if they do not meet this requirement, do not have an active registration status, or have active exclusions. Selecting a compliant vendor will download PDF record of their compliance.
This tool's search capability uses the openGSA SAM.gov Entity Management API. Vendors that have selected "DOES NOT" for both FAR 52.204-26(c)(1) and (2) are marked as compliant. A vendor is not selectable if they do not meet this requirement, do not have an active registration status, or have active exclusions. Selecting a compliant vendor will download PDF record of their compliance.

- https://open.gsa.gov/api/entity-api/
[You can find more information about the SAM.gov's API here](https://open.gsa.gov/api/entity-api/).

Search results omit entities without representations and certifications, which are those with a "purpose of registration code" of "Federal Assistance Awards" only (Code Z1) and child entities (non-zero/non-null EFT Indicator).

## Objective
### Objective

The purpose of this tool is to enable the broadest possible user-base, including non-procurement-experts, the ability to determine vendor 889 compliance from their SAM.gov record as quickly as possible and with little or no training.

## Libraries

The 889 Representations SAM.gov Tool is written with the FastAPI Python framework and uses a Vue.js front-end. PDFs are generated in the browser using jsPDF.

- FastAPI https://fastapi.tiangolo.com (MIT)
- Vue.js https://vuejs.org (MIT)

Other python libraries include:

- httpx https://www.python-httpx.org/ (BSD) -- An Async HTTP library. -- Used to make calls the SAM.gov Entities API in python without blocking.
- jsPDF https://parall.ax/products/jspdf (MIT) -- Make PDFs with Javascript. -- Used to generate PDF records of vendor 889 compliance on the fly without additional calls to the backend

The following libraries from requirements.dev.txt are not required for running a production instance, but may be useful in development:

- pytest https://github.com/pytest-dev/pytest/ (MIT) -- Makes it easy to write small, readable tests, and can scale to support complex functional testing for applications and libraries. -- Used to help write and execute tests.
- pylint https://pylint.pycqa.org/en/latest/ (GPL2) -- Pylint is a static code analyzer -- Used to help in writing clean consistent code

The python FastAPI application can be deployed in a production environment using a variety of tools. While they are not required, these tools may be useful:

- gunicorn https://docs.gunicorn.org/en/stable/ (MIT) -- WSGI HTTP Server for UNIX -- Used to run FastAPI app
- uvicorn https://www.uvicorn.org (BSD) -- Provides asynchronous workers to allow gunicorn to handle asyncIO
Additional libraries used by the tool can be found in requirements.txt.

## Features

The tool performs two main tasks.

1. Search term pre-processing. - Improves the quality of search results returned by the SAM.gov Entities API by modifying the search expression provided by the user. Regular expressions are used to identify SAM.gov UEIs, and US and NATO cage codes. See the search_preprocessor.py file and associated tests in tests/test_search_preprocessor.py
2. Determine entity compliance status. - Call the SAM.gov Entities API and append the response data with a "samToolsData" section for each vendor containing compliance information. See compliance/compliance_rules.py for compliance rules and associated tests in tests/test_compliance_rules.py.

## API Endpoints
### API Endpoints

The 889 Representations SAM.gov Tool provides a single API endpoint. This endpoint allow for other tools (like the Vue.js front-end) that require 889 compliance data from SAM.gov to obtain this from the tool's. The endpoint is defined in `__init__.py`.

Expand All @@ -61,71 +44,63 @@ Example:

`<HOST_URL>/api/entity-information/v3/entities?samToolsSearch=mcmaster&includeSections=[samToolsData,entityRegistration,coreData]&registrationStatus=A`

## Code structure

- `__init__.py` --> Main FastAPI application
- samtools/compliance --> Objects for determining compliance from SAM.gov data
- samtools/sam_api --> Run search preprocessor, call SAM.gov Entities Management API, append compliance data to response
- tests --> Tests

Tests can be run using pytest:
## Development and Testing ##
### Prerequisites ###
Before beginning, ensure that you have Python >= 3.8 and that LTS version of Node.js installed on your sytem.

Additionally, you **will need to register for a SAM.gov API key**. [You can find more information here](https://open.gsa.gov/api/entity-api/).

`pytest tests/test_entity_information.py`
Once you have a key, you will need to set the corresponding environment variable locally, for example:

`etc...`

---

## Local development installation instructions

### Clone the repository into a directory

```
git clone <CODE_REPOSITORY>
cd samtools
```

### Install python >= 3.8, pip, and virtualenv using apt-get, brew, etc.

This is an example on systems that use apt-get (such as Debian-based Linux) commands must be run as root or with superuser privileges (sudo). If developing locally on a Mac or Windows machine, take appropriate steps to ensure you have Python version 3.8 installed.

Example: `sudo apt-get update`

```
apt-get update
apt-get install -yq git python3 python3-pip
pip3 install --upgrade pip virtualenv
export SAM_API_KEY=YOUR_API_KEY
```

### Create a python virtual environment
### Setting up the backend (Python/FastAPI) environment ###
#### Create a python virtual environment

The SAM.gov Tool comes with a bash script that automates several of the build steps. Alternatively, you can look at the contenst of the script and run the commands as desired. It is just building a python virtual-env and installing dependencies:

```
cd samtools
bash build_samtools.sh
source venv/bin/activate
```

Then install the development dependencies:
```
pip install -r requirements.dev.txt # install development-only python requirements
```

### Setting up the frontend (Node.js/Vue.js) environment ###
Install the required Node.js dependencies:
```sh
npm install
```

### Setup instance-specific data (SAM.gov Entity Management API key and contact email)

To communicate with the SAM.gov API you need an API Key. You will need to set an environmental variable with this key. On Mac/Linux systems you can eport it:

### Quickstart for dev ###
To run both the backend and frontend at the same time with a single command:
```sh
npm run dev
```
export SAM_API_KEY=YOUR_API_KEY

To run just the backend:
```sh
npm run dev:backend
```

And to run just the frontend:
```sh
npm run dev:frontend
```

### Run the FastAPI application using uvicorn. The `--reload` will watch for changes and reload the app.
### Running the backend manually ###

#### Run the FastAPI application using uvicorn. The `--reload` will watch for changes and reload the app.

```
uvicorn samtools.wsgi:app --reload
```

### Optional: use gunicorn as a web server gateway interface (WSGI)
#### Optional: use gunicorn as a web server gateway interface (WSGI)

This is not required to run a development instance of FastAPI application, but is the recommended way to run in production.
If you would like to run gunicorn with uvicorn workers (for nice async IO):
Expand All @@ -135,27 +110,78 @@ gunicorn samtools.wsgi:app --workers 2 --worker-class uvicorn.workers.UvicornWor
```

NOTE: gunicorn runs on port 8000 by default.

### Running Tests ###
#### Backend Tests ####
To quickly run the backend tests:
```sh
npm run test:backend
```

#### Frontend Tests ####
To quickly run basic frontend tests:
```sh
npm run test:frontend
```

##### Run Unit Tests with [Vitest](https://vitest.dev/)

```sh
npm run test:unit
```

##### Run End-to-End Tests with [Cypress](https://www.cypress.io/)
**NOTE: the framework stubs these e2e tests by default, but we do not have any e2e test cases currently**. We are keeping the capability here for potential future use.
```sh
npm run build
npm run test:e2e # or `npm run test:e2e:ci` for headless testing
```

##### Lint with [ESLint](https://eslint.org/)

### That's it!
```sh
npm run lint
```

### Additional frontend commands ###
#### Compile and Hot-Reload for Development

Hopefully that all went smoothly and now you can continue to develop and improve the SAM.gov tool on your local machine!
```sh
npm run dev
```

---
#### Compile and Minify for Production

```sh
npm run build
```

#### Build USWDS assets:
The site uses USWDS for most styling rather than in-component styles. This may not be the right choice; we should iterate and find the technique that balances ease-of-use with maintainability.

``` sh
npx gulp compile
```

This will build css files from sources in `src/scss` and `node_modules/@uswds` and deposit results in `src/assets`.

See `gulpfile.js` for settings.

## Production deployment
## Production Deployment
### Frontend (Cloud.gov Pages)
Because the Vue simply builds a static html/javascript site, it can run from any system that can serve static content. This includes Cloud.gov-Pages (Federalist). The current configuration in `vite.config.js` outputs build artifacts to `_site` where federalist expects to find them. `package.json` includes a `federalist"` script that runs `vite build`. Cloud.gov Pages will watch the github repo for changes on the main branch and re-build files when it changes.

### Deploying to cloud.gov & cloud.gov pages
This repository contains code for a front-end Vue.js application. This can be built and served to any environment that can serve static HTML/CSS/Javascript pages.
Currently, using Cloud.gov requires the use of the `createWebHashHistory` style urls. This allows the Vue Router to use URLS that include `#` like `#/search/some_company/2` without trying to load a page at a different path. This allows users to refresh the page or share a link.

#### Serving the front
[Cloud.gov pages](https://pages.cloud.gov/sites) expects a repository with a build script. See [their documentation](https://cloud.gov/pages/documentation/) for detailed instruction on setting this up. This project should select the [node.js engine](https://cloud.gov/pages/documentation/node-on-pages/), which will look for a `federalist` script in package.json. The front end needs to know the URI of the backend api. This can be set up on per-environment basis, in the `front-end/.env.xxx` files. The script `front-end/bin/build.sh` determines the mapping between branches and env files.
#### Environment variables
The production build will expect to find an env `VITE_API_DOMAIN` pointing to the 889 tool API. This is currently running on cloud.gov.

#### Serving the backend on cloud.gov
### Backend (FastAPI on Cloud.gov)

##### Create a cloud.gov instance
#### Create a cloud.gov instance
Your cloud.gov account must have the `SpaceDeveloper` role in each space in order to run these scripts.

##### Bootstrap the cloud.gov environment
#### Bootstrap the cloud.gov environment
Before the first deployment, you need to run the bootstrap script, where `SPACE` is one of `dev`, `test`, `staging`, or `prod`. This will create all the necessary services that are required to deploy the app in that space.

```
Expand All @@ -168,7 +194,7 @@ You can monitor the services deployment status with `cf services`. It can take q
bin/cg-bootstrap-app.sh SPACE
```

##### Create cloud.gov service accounts
#### Create cloud.gov service accounts
*Note: Only one service account is needed for a cloud.gov space. We don't need to do this if there is an existing service*
Create a service account for each space. These accounts will be used by GitHub Actions to deploy the app. Since we are currently manually deploying to the `test` space, we do not need a service account for that space.

Expand All @@ -178,7 +204,7 @@ bin/cg-service-account-create.sh SPACE

Take note of the username and password it creates for each space.

##### Configure the GitHub environments
#### Configure the GitHub environments

1. [Create environments in the GitHub repository](https://github.com/GSA/889-tool/settings/environments) that correspond with each space that GitHub Actions will deploy to (i.e., `dev`, `staging`, and `prod`)
2. Within each GitHub environment, configure:
Expand Down
27 changes: 27 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Security Policy

As a U.S. Government agency, the General Services Administration (GSA) takes
seriously our responsibility to protect the public's information, including
financial and personal information, from unwarranted disclosure.

Software developed by the U.S. General Services Administration (GSA)
is subject to the [GSA Vulnerability Disclosure Policy](https://gsa.gov/vulnerability-disclosure-policy).

Please consult our policy for:
* How to submit a report if you believe you have discovered a vulnerability.
* GSA's coordinated disclosure policy.
* Information on how you may conduct security research on GSA developed
software and systems.
* Important legal and policy guidelines.

## Supported Versions

Please note that only certain branches are supported with security updates.

| Version (Branch) | Supported |
| ---------------- | ------------------ |
| main | :white_check_mark: |
| other | :x: |

When using this code or reporting vulnerabilities please only use supported
versions.
Loading

0 comments on commit ae042f2

Please sign in to comment.