Roadmap planning for the CA SDK - Topics for discussion and availability

[Willmish]

Getting back on topic of having a roadmap planned for the CA SDK, during Green-Software-Foundation/carbon-aware-sdk#398 we agreed to kick this off with a discussion here, to get people's availability (roughly, in weeks/months), and to collect topics that people think important to discuss.

• E.g. - easy to pick up samples are necessary for the repo (@vaughanknight ) - from experience CA SDK is not easy to set up
• E.g. - what we want to add to the project's API - any new endpoints necessary, any specific handling of endpoints ([Feature Contribution]: Better to throw exception when future datetime is specified to endpoints for current/historical data carbon-aware-sdk#396)
• Establishing a release schedule, and planning of feature introduction in future releases (@YaSuenag)

Idea is to:

• Collect requests/ideas from the contributors of the project
• discuss them internally by the developers and prioritise
• Present the list to the Open Source WG and collect feedback on priotisation

Please leave your thoughts and comments down below!

[YaSuenag]

I think we need to release schedule or release cadence of the SDK.
As a most recent schedule, I'd like to propose in below:

• Next version: v1.2
• Release date: end of 2023
• New features:
  • [Feature Contribution]: Data caching in the SDK carbon-aware-sdk#397
  • Add Helm chart and workflow carbon-aware-sdk#381
  • Data exporter for Prometheus
    • It might be better to migrate to .NET 8 because it looks like having useful APIs to implement. https://learn.microsoft.com/en-us/dotnet/core/diagnostics/metrics-instrumentation
  • [Bug]: Fix and update doc deployment workflow carbon-aware-sdk#344
    • And also we need to enhance documents for users.

IMHO following proposals should be useful for the SDK users include other GSF projects (e.g. IEF), so they should be introduced soon (v1.3 or later?)

• Add 2b-generate-java-client.yaml carbon-aware-sdk#359
• Publish client library for Python to PyPi

In future, following issues shuld be available.

• [Feature Contribution]: Multiple data source support carbon-aware-sdk#345
• [Feature Request]: Missing option to select WattTime model version (Auditability) carbon-aware-sdk#260

We should set milestone to each issues/PRs when we decide a roadmap.

[Willmish]

Adding to the release schedule point:

• Maybe we should consider expanding project roles to decrease the bottleneck of me/@vaughanknight on releases + allow others to block releases when deemed not ready

[bderusha]

Especially for open source projects I have found that it is better to have EITHER schedule-based versions OR feature bundle versions, but not both.

I think a schedule-based versioning plan (for example, cutting a new release each quarter) along with a roadmap of priorities (but not a commitment to which features are tied to any specific version) can be a powerful combination.

[danuw]

For me (and partly things I have open) are:

• finish release process / DevOps tasks including docusaurus (I have a couple of
• add more samples and encourage more blog posting about how to use?

[YaSuenag]

add more samples

So Green-Software-Foundation/carbon-aware-sdk#340 can be included in next release? I believe this PR is useful to demonstrate the SDK on Windows / Linux because easy to setup.

[Willmish]

@bderusha could you please add your thoughts on this from our discussion on Friday (13.10.2023)?

[bderusha]

I propose adding "Publishing the SDK as a NuGet package" be included on the roadmap.

There are a few things I would like to see happen as part of such work:

• Review + Merge the ADR-0006 implementation Implement Data Source Registration per ADR-0006 carbon-aware-sdk#366
• Update ADR-0014 to include a detailed proposal for how to split the project into packages
• Implementation of ADR-0014 to structure the code to support packaging
• Implementation of package publishing pipelines

Benefits:

• Creating a published "SDK package" opens up the SDK to a wider audience by making it easier to integrate into existing software, thereby increasing the impact of the project.
• Creating a "core package" allows community members to create their own DataSources without needing to clone the entire project.

[YaSuenag]

+1 to publish NuGet package, but the package (CASDK library) and OpenAPI client library need to be distinguished. I think it is better to publish both (to GitHub Package) in next release, but we need to think naming convention I think.

[bderusha]

Totally agree. I put "a detailed proposal for how to split the project into packages" as its own line item for exactly that reason. I don't think the project should be published as-is right now. It wasn't structured with packaging as a goal and thus will need a little reorganization to be effective.

[bderusha]

I have also gotten anecdotal feedback from a number of users that configuration of the SDK is challenging. It may be worth soliciting proposals for improvements to the configuration approach to make it more user friendly.