Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quota Page: Show EC2 Instance Type availability per Region+AZ #76

Open
wants to merge 15 commits into
base: dev
Choose a base branch
from
Open

Quota Page: Show EC2 Instance Type availability per Region+AZ #76

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
3f46969
Update Spot Instance offerings API call
neilramsay Oct 25, 2022
f1e4000
FamilyRegions variable include InstanceType per AZ
neilramsay Oct 25, 2022
1652656
Update Quota controller to pass Region and AZs
neilramsay Oct 25, 2022
b89b74c
Create keys Angular filter for finding Object.keys(obj)
neilramsay Oct 27, 2022
bf03782
New Campaign page: Update Family Regions usage
neilramsay Oct 28, 2022
db18305
Quota page: Fixup available regions to handle Region[AZ] object
neilramsay Oct 27, 2022
6f5bd34
Quota page: Display AZ capacity for instance type
neilramsay Oct 25, 2022
f718a4d
PricingService: Update FamilyRegions usage
neilramsay Oct 27, 2022
6b297a5
Merge pull request #78 from c6fc/dev
c6fc Dec 9, 2022
0b6775b
Merge pull request #84 from c6fc/dev
c6fc May 5, 2023
4012807
Merge pull request #92 from c6fc/dev
c6fc Dec 8, 2023
55044df
Merge pull request #95 from c6fc/dev
c6fc Jan 10, 2024
7a29e7f
Merge pull request #96 from c6fc/dev
c6fc Jan 12, 2024
fdcc282
Merge pull request #99 from c6fc/dev
c6fc Apr 6, 2024
f60ad85
Merge branch 'c6fc:main' into feature/instance_availability
neilramsay Sep 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 24 additions & 14 deletions bin/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,26 +224,36 @@ async function getAZsWithQuota() {

console.log("[+] Retrieved quotas.");

const instanceRegions = {};
const instanceTypes = Object.keys(families).reduce((instances, family) => {
const firstInstance = Object.keys(families[family].instances)[0];
instances[firstInstance] = family;
instanceRegions[family] = [];

// Map instance types to GPU
const instanceTypesToGpu = Object.keys(families).reduce((instances, gpu) => {
Object.keys(families[gpu].instances).forEach(instanceType => {
instances[instanceType] = gpu
});
return instances;
}, {});
}, {})

const instanceAvailabilityInRegions = {};
const offeringsPromises = regions.reduce((offerings, region) => {
const ec2 = new aws.EC2({ region });

offerings.push(ec2.describeInstanceTypeOfferings({
LocationType: "region"
LocationType: "availability-zone",
Filters: [
{
Name: "instance-type",
Values: Object.keys(instanceTypesToGpu)
}
]
}).promise().then((data) => {

const instances = data.InstanceTypeOfferings
.filter(e => Object.keys(instanceTypes).includes(e.InstanceType))
.map(e => {
instanceRegions[instanceTypes[e.InstanceType]].push(region);
data.InstanceTypeOfferings
.map(offering => {
const gpu = instanceTypesToGpu[offering.InstanceType];

// Build up path
instanceAvailabilityInRegions[gpu] ??= {};
instanceAvailabilityInRegions[gpu][region] ??= {};
instanceAvailabilityInRegions[gpu][region][offering.InstanceType] ??= [];
instanceAvailabilityInRegions[gpu][region][offering.InstanceType].push(offering.Location);
});
}).catch(e => {
console.log(`[-] Unable to get instance support for ${region}, but this isn't fatal.`);
Expand All @@ -255,7 +265,7 @@ async function getAZsWithQuota() {

await Promise.all(offeringsPromises);

result.familyRegions = instanceRegions;
result.familyRegions = instanceAvailabilityInRegions;

console.log("[+] Retrieved per-region instance support.");

Expand Down
5 changes: 5 additions & 0 deletions site-content/angular/app.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ angular
.filter(e => mask.includes(e[field]));
}
})
.filter('keys', function() {
return function(obj) {
return Object.keys(obj)
}
})
.filter('equals', function() {
return function(items, equals) {
if (typeof items != "object") {
Expand Down
2 changes: 1 addition & 1 deletion site-content/angular/controllers/npkMainCtrl.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2622,7 +2622,7 @@ angular
$scope.quotas = QUOTAS;
$scope.families = FAMILIES;
$scope.familyregions = FAMILYREGIONS;
$scope.all_regions = Object.keys(REGIONS);
$scope.all_regions = REGIONS;

$scope.onReady = function() {
$scope.$parent.startApp();
Expand Down
2 changes: 1 addition & 1 deletion site-content/angular/services/pricingSvc.js
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion site-content/views/new-campaign.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@
Region
</div>
<div class="card-body bg-dark p-1 pr-3">
<button ng-click="selectRegion(cost)" ng-repeat="cost in selectedFamily.cost | fieldMemberOf:'region':familyRegions[selectedFamily.gpu] | orderBy:'quota':true" class="btn btn-sm btn-block waves-effect waves-light m-1" ng-class="[
<button ng-click="selectRegion(cost)" ng-repeat="cost in selectedFamily.cost | fieldMemberOf:'region':(familyRegions[selectedFamily.gpu]|keys) | orderBy:'quota':true" class="btn btn-sm btn-block waves-effect waves-light m-1" ng-class="[
{ 'btn-warning': (selectedRegion == cost && cost.quota < 16) },
{ 'btn-outline-warning': (selectedRegion != cost && cost.quota < 16) },
{ 'btn-success': (selectedRegion == cost && cost.quota >= 16) },
Expand Down
38 changes: 24 additions & 14 deletions site-content/views/quota.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
</div>
<div>
<small>
Regions with zero quota are not shown. Configured regions: {{ all_regions.join(', ') }}.
Regions with zero quota are not shown. Configured regions: <span ng-repeat="(region, azs) in all_regions">{{ region }}<span ng-if="!$last">, </span></span>.
</small>
</div>
</div>
Expand All @@ -24,23 +24,23 @@
<tr class="text-dark bg-white border border-white">
<th class="border border-dark" width="200">GPU</th>
<th class="border border-dark" width="170">Instance</th>
<th class="border pt-2" width="150" ng-class="!familyregions[gpu].includes(region) ? 'bg-dark text-white border-white' : 'bg-white text-dark border-dark'" ng-repeat="region in all_regions" style="line-height: 1.2;">
<th class="border pt-2" width="150" ng-class="!familyregions[gpu][region] ? 'bg-dark text-white border-white' : 'bg-white text-dark border-dark'" ng-repeat="(region, azs) in all_regions" style="line-height: 1.2;">
{{ region }}
<br>
<div ng-if="familyregions[gpu].includes(region) && !!quotas[region][families[gpu].quotaCode]" class="badge badge-sm badge-dark mb-2">
<div ng-if="familyregions[gpu][region] && !!quotas[region][families[gpu].quotaCode]" class="badge badge-sm badge-dark mb-2">
{{ quotas[region][families[gpu].quotaCode] }} vCPU
</div>
<div ng-if="familyregions[gpu].includes(region) && !quotas[region][families[gpu].quotaCode]" class="badge badge-sm badge-danger mb-2">
<div ng-if="familyregions[gpu][region] && !quotas[region][families[gpu].quotaCode]" class="badge badge-sm badge-danger mb-2">
No Quota
</div>
<div ng-if="!familyregions[gpu].includes(region)" class="badge badge-sm badge-secondary mb-2">
<div ng-if="!familyregions[gpu][region]" class="badge badge-sm badge-secondary mb-2">
Unsupported
</div>
</th>
</tr>
</thead>
<tbody on-finish-render>
<tr class="text-white border border-white" ng-repeat="family in families[gpu].instances | toArray | orderBy: '[1]'">
<tr class="text-white border border-white" ng-repeat="instanceType in families[gpu].instances | toArray | orderBy: '[1]'">
<td ng-if="$first" class="text-center border border-white p-3" style="line-height: 0.8; font-variant: small-caps;" rowspan="10">
<b>{{ gpu }}</b>
<br><br>
Expand All @@ -51,23 +51,33 @@
</small>
</td>
<td class="text-center p-2" style="line-height: 1.3;">
<b>{{ family._id }}</b>
<b>{{ instanceType._id }}</b>
<br>
<small>GPUs: {{ instanceType[0] }}; vCPUs: {{ instanceType[1] }}</small>
<br>
<small>GPUs: {{ family[0] }}; vCPUs: {{ family[1] }}</small>
</td>
<td class="border border-white p-2 text-center" ng-repeat="region in all_regions">
<div ng-if="familyregions[gpu].includes(region) && !quotas[region][families[gpu].quotaCode]" class="text-danger" style="line-height: 1;">
<td class="border border-white p-2 text-center" ng-repeat="(region, azs) in all_regions">
<div ng-if="familyregions[gpu][region] && !quotas[region][families[gpu].quotaCode]" class="text-danger" style="line-height: 1;">
<small>Quota is zero</small>
</div>
<div ng-if="familyregions[gpu].includes(region) && quotas[region][families[gpu].quotaCode] > 0 && quotas[region][families[gpu].quotaCode] < family[1]" class="text-warning" style="line-height: 1;">
<div ng-if="familyregions[gpu][region] && quotas[region][families[gpu].quotaCode] > 0 && quotas[region][families[gpu].quotaCode] < instanceType[1]" class="text-warning" style="line-height: 1;">
<small>Insufficient quota</small>
</div>
<div ng-if="familyregions[gpu].includes(region) && quotas[region][families[gpu].quotaCode] >= family[1]" class="text-warning" style="line-height: 1;">
<div class="btn btn-success">{{ (quotas[region][families[gpu].quotaCode] / family[1]).toString().split('.')[0] }} Instances</div>
<div ng-if="familyregions[gpu][region] && quotas[region][families[gpu].quotaCode] >= instanceType[1]" class="text-warning" style="line-height: 1;">
<div class="btn btn-success">{{ (quotas[region][families[gpu].quotaCode] / instanceType[1]).toString().split('.')[0] }} Instances</div>
</div>
<div ng-if="!familyregions[gpu].includes(region)" class="text-secondary" style="line-height: 1;">
<div ng-if="!familyregions[gpu][region]" class="text-secondary" style="line-height: 1;">
<small>Not supported<br>by this region</small>
</div>
<div ng-if="familyregions[gpu][region][instanceType._id].length == azs.length" class="badge badge-sm badge-success mb-2" data-placement="top" title="All AZs supported; launch times should be quick">
{{ familyregions[gpu][region][instanceType._id].length }}/{{ azs.length }} AZs
</div>
<div ng-if="familyregions[gpu][region][instanceType._id].length > 1 && familyregions[gpu][region][instanceType._id].length < azs.length" class="badge badge-sm badge-warning mb-2" data-placement="top" title="Two or more AZs supported; launch times may take longer">
{{ familyregions[gpu][region][instanceType._id].length }}/{{ azs.length }} AZs
</div>
<div ng-if="familyregions[gpu][region][instanceType._id].length == 1" class="badge badge-sm badge-danger mb-2" data-toggle="tooltip" data-placement="top" title="Only one AZ supported; launch times may be quite long">
{{ familyregions[gpu][region][instanceType._id].length }}/{{ azs.length }} AZs
</div>
</td>
</tr>
</tbody>
Expand Down