-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow data sources to be synced from the UI #379
Allow data sources to be synced from the UI #379
Conversation
✅ Deploy Preview for ami-web canceled.
|
✅ Deploy Preview for ami-storybook canceled.
|
…8-Allow-data-sources-to-be-synced-from-the-UI
Created a new issue Sign up button #385 in GitHub |
…ces-to-be-synced-from-the-UI
a84584b
to
88bc9f1
Compare
FE code wise I'm still using some hard coded strings, I feel we are still iterating a bit on the UI here. Do you think we are showing too much details about the connection status now (making the view a bit cluttered), or is ok? |
@annavik Thank you for the comments and observations. I agree the UI is looking a but cluttered, but I think it is necessary while we work out the kinks. I want to provide all the info we need to troubleshoot. Especially because syncing data can take a long time, and you may not notice an error right away. So it's great that we are making it possible to check the status of everything ahead of time that may break the sync.
|
… github.com:RolnickLab/ami into AMI-278-Allow-data-sources-to-be-synced-from-the-UI
…-synced-from-the-UI
I just fixed the last large todo for this mega PR (related to data sync job status and interfering with the ML job status). I will return to the smaller remainders asap. So close to merging this one! |
Great job @mihow ! 🙏 |
Bugs to fix:
Discovered bugs, may no longer be relevant:
Several issues raised during this PR have been moved to related issues:
#400
#455
#310
#456
#236
Configure S3 storage source for a project
![image](https://private-user-images.githubusercontent.com/158175/338846720-9009850a-5c08-45e6-bc90-803f687045d1.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkyODczNDIsIm5iZiI6MTczOTI4NzA0MiwicGF0aCI6Ii8xNTgxNzUvMzM4ODQ2NzIwLTkwMDk4NTBhLTVjMDgtNDVlNi1iYzkwLTgwM2Y2ODcwNDVkMS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIxMVQxNTE3MjJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0xZGEzMGI5Mjg3ZWUxMzRjYmE5N2NmZDlmNTZhYTMyYWIyN2M2NTg4Y2Q2Zjg3NWFmMDhkODA3ODdkNmU2Mjg0JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.5xzM4u-mzU81ttl2sai18MK6xwscnC-eWee0AbK5YOU)
Test the connection, see which deployments are using this storage
![image](https://private-user-images.githubusercontent.com/158175/338846894-8d63c5e3-0571-4bf0-846f-ec4af1ab96ae.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkyODczNDIsIm5iZiI6MTczOTI4NzA0MiwicGF0aCI6Ii8xNTgxNzUvMzM4ODQ2ODk0LThkNjNjNWUzLTA1NzEtNGJmMC04NDZmLWVjNGFmMWFiOTZhZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIxMVQxNTE3MjJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1jZTRmNGIxMmVmYTIxNTY2NmRiNWIxNGY4NGU2MWI2ZWYzZGE0MDZkMjMxNDk1ZWQ2YjUwOTBjNWI3M2E0ZjFiJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.6j8VlWSPofoBlAhmSO1_CiS5V0oUCgs9ExwtqkivFcM)
Configure the subdirectory for each deployment & test the config
![image](https://private-user-images.githubusercontent.com/158175/338846266-5ad8fd64-6b7f-4cb3-9d73-597447c36be0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkyODczNDIsIm5iZiI6MTczOTI4NzA0MiwicGF0aCI6Ii8xNTgxNzUvMzM4ODQ2MjY2LTVhZDhmZDY0LTZiN2YtNGNiMy05ZDczLTU5NzQ0N2MzNmJlMC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIxMVQxNTE3MjJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02ZmNjYWE0ODA2NzZiZjhmZGJmYjRiMTk1MGFjNmRhOGRmNWZiZWFiMWJkZmE2NzAwNGFkNjU2YWVlZDBhMzljJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.6twFsK8BaqRgLIC2yps0kkqco3BkzKR2OHWG4IZ2LvU)
Start the sync
![image](https://private-user-images.githubusercontent.com/158175/338847282-d6454b4c-549f-4473-91cb-f09f7acc928d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkyODczNDIsIm5iZiI6MTczOTI4NzA0MiwicGF0aCI6Ii8xNTgxNzUvMzM4ODQ3MjgyLWQ2NDU0YjRjLTU0OWYtNDQ3My05MWNiLWYwOWY3YWNjOTI4ZC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIxMVQxNTE3MjJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iODNhYTVjOTkyYzhlYzdhOWM4NGI1ZDBkZmMyYmU1ZDZhZjFiZTNiMjI0MGE5ODFkN2JjZDM5OTZlYzkzYTgzJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.Lpq2v10GYZOpO9ODyAjl6ho6ixu_vAeQEGAD4Qw0KoU)
View status of sync job
![image](https://private-user-images.githubusercontent.com/158175/338847359-63d323a3-cfae-4f94-a6e9-014d17572a17.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkyODczNDIsIm5iZiI6MTczOTI4NzA0MiwicGF0aCI6Ii8xNTgxNzUvMzM4ODQ3MzU5LTYzZDMyM2EzLWNmYWUtNGY5NC1hNmU5LTAxNGQxNzU3MmExNy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIxMVQxNTE3MjJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1jZWRhMzQ5ZjNkNDU5OGNlZDA4ZWFjYTI0NGEzMjlkNzIyYzk3Y2Y4MjVlZDk2ZTNlNGI5NjFiMGM1NTM5Yzg4JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.01h_LisGuskFuf9Ulf6N-qXzLGKaHO-assTmf4kZ4EU)
See the number of captures increase for the deployment
![image](https://private-user-images.githubusercontent.com/158175/338847463-05e4de1c-3d8b-4d8b-a25b-5fa7eeacff93.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkyODczNDIsIm5iZiI6MTczOTI4NzA0MiwicGF0aCI6Ii8xNTgxNzUvMzM4ODQ3NDYzLTA1ZTRkZTFjLTNkOGItNGQ4Yi1hMjViLTVmYTdlZWFjZmY5My5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIxMVQxNTE3MjJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT00YWJhNzYzNzE2Y2FiNjNjNjY2N2IzNDZhYjg3ZGI0OTc4Nzg5NDM5YTcxMWVjYzgwZTRiZWFhOGRhOWM4NThmJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.ExDliW63utg6jyRNBTqTXInyBbQAw7Yid_FHxhwZRug)