[Discovery] Understand how Hosted Devstack builds its images

[jristau1984]

Dig thru the Hosted Devstack code ( and ask Justin Hynes for help as well), to understand how the images are built.
Qs:

• Does it do edxapp only, or IDAs as well?
• How long does it take to build?
• What sample data gets seeded in the image (edxapp, IDAs, etc)?
• How are MFEs pre-configured in the image?

[jristau1984]

The best channel to start this discovery would be #tech-hosted-devstack. The best person to reach out to would be Justin Hynes.

[mumarkhan999]

👉 Does it run edxapp only, or IDAs as well?

Hosted Devstack runs the default set of IDAs found in a "regular" Devstack setup—edx-platform, discovery, ecommerce, credentials—and includes some configuration to run enterprise services as part of edx-platform as well.

The MFEs enabled by default are listed here:
https://2u-internal.atlassian.net/wiki/spaces/ENG/pages/1020362766/CHANGELOG#V29(2024-11-26)

👉 How long does it take to build?

When you launch an EC2 instance of Hosted Devstack, the latest snapshot of the AMI is already selected for you. This launch process typically takes 2 to 3 minutes. However, the initial creation of a new Hosted Devstack instance may experience some lag during the first 30-60 minutes after creation.

It is very common for a brand-new Hosted Devstack instance to be slow immediately after launch. Typically, the Django services will be up and running before any of the MFEs, as the MFEs need to start up and download all their dependencies while the EC2 instance is still completing its initialization. The initial slowness usually resolves within the first hour after launching a new Hosted Devstack instance.

👉 What sample data gets seeded in the image (edxapp, IDAs, etc.)?

A dedicated AMI has been gradually customized since its initial creation. Occasionally (once every 1-2 months), the owning team refreshes the base AMI and makes adjustments or adds new configurations as needed. The team has been tracking these changes in Confluence:
https://2u-internal.atlassian.net/wiki/spaces/ENG/pages/1020362766/CHANGELOG.

👉 How are MFEs pre-configured in the image?

To pre-configure an MFE, we update the site configuration, waffle flags, and the DEFAULT_SERVICES variable in the options.local.mk file. Then, we take a snapshot of this running instance. Any new instances launched using this snapshot will have the MFE pre-configured.

More details on how to add these configurations are available here:
https://2u-internal.atlassian.net/wiki/spaces/ENG/pages/1020362766/CHANGELOG#V29(2024-11-26)

Folks with pre-provisioned Hosted Devstack instances need to make updates manually (or create a new Hosted Devstack instance).

[iamsobanjaved]

Closing this as all the details we wanted are added to the issue.