Skip to content

Latest commit

 

History

History
365 lines (267 loc) · 14.1 KB

pull-requests.md

File metadata and controls

365 lines (267 loc) · 14.1 KB

Pull requests

Setting up your local environment

To get started, you will need to have git installed locally. Depending on your operating system, there are also a number of other dependencies required.

Enable WSL2 on your machine.

Install CUDA from Nvidia installation guide

Step 1: Fork

Fork the project on GitHub and clone your fork locally.

git clone [email protected]:username/node.git
cd node
git remote add upstream https://github.com/jazelly/FinetuneLLMs.git
git fetch upstream

Configure git so that it knows who you are:

git config user.name "Random User"
git config user.email "[email protected]"

You can use any name/email address you prefer here. We only use the metadata generated by git using this configuration for properly attributing your changes to you in the AUTHORS file and the changelog.

If you would like for the GitHub UI to link the commit to your account and award you the Contributor label after the changes have been merged, make sure this local email is also added to your GitHub email list.

After forking and cloning, you can follow the build guide to setup dev environment.

Step 2: Branch

As a best practice to keep your development environment as organized as possible, create local branches to work within. These should also be created directly off of the upstream default branch.

git checkout -b my-branch -t upstream/HEAD

The process of making changes

Step 3: Code

Pull requests in FinetuneLLMs typically involve changes to one or more of a few places in the repository.

  • Frontend code contained in the frontend directory
  • Node.js server code contained in the server directory
  • Python trainer code contained in the trainer directory
  • Documentation in doc/api

Step 4: Commit

It is a best practice to keep your changes as logically grouped as possible within individual commits. There is no limit to the number of commits any single pull request may have, and we find it easier to review changes that are split across multiple commits.

git add my/changed/files
git commit

Multiple commits often get squashed when they are landed.

Commit message guidelines

A good commit message should describe what changed and why.

  1. The first line should:

    • contain a short description of the change
    • be entirely in lowercase with the exception of proper nouns, acronyms, and the words that refer to code, like function/variable names
    • be prefixed with the name of the changed subsystem and start with an imperative verb. Check the output of git log --oneline files/you/changed to find out what subsystems your changes touch.

    Examples:

    • ui: refactor sidebar icons
    • trainer: idle worker is closed within 10 secs

  2. Keep the second line blank.

  3. If your patch fixes an open issue, you can add a reference to it at the end of the log. Use the Fixes: prefix and the full issue URL. For other references use Refs:.

    Examples:

    • Fixes: https://github.com/jazelly/FinetuneLLMs/issues/5
    • Refs: https://github.com/jazelly/FinetuneLLMs/pull/2

Sample complete commit message:

trainer: idle worker is closed within 10 secs

The body of the commit message should be one or more paragraphs, explaining
things in more detail. Please word-wrap to keep columns to 72 characters or
less.

Fixes: https://github.com/jazelly/FinetuneLLMs/issues/5

Step 5: Rebase

As a best practice, once you have committed your changes, it is a good idea to use git rebase (not git merge) to synchronize your work with the main repository.

git fetch upstream HEAD
git rebase FETCH_HEAD

This ensures that your working branch has the latest changes from nodejs/node.

Step 6: Test

It is recommended to test your changes before further actions. You can test it by performing manual testing. For now, FinetuneLLMs does not have a CI flow, but it is WIP.


### Step 7: Push

Once you are sure your commits are ready to go, with passing manual tests and linting,
begin the process of opening a pull request by pushing your working branch to
your fork on GitHub.

```bash
git push origin my-branch

You may find git push is failed after rebasing. It is normal and you can do

git push --force-with-lease origin my-branch

Step 8: Opening the pull request

From within GitHub, open a new pull request.

Once opened, pull requests are usually reviewed within a day.

To get feedback on your proposed change even though it is not ready to land, use the Convert to draft option in the GitHub UI. Do not use the wip label as it might not prevent the PR from landing before you are ready.

Step 9: Discuss and update

You will probably get feedback or requests for changes to your pull request. This is a big part of the submission process so don't be discouraged! Some contributors may sign off on the pull request right away, others may have more detailed comments or feedback. This is a necessary part of the process in order to evaluate whether the changes are correct and necessary.

To make changes to an existing pull request, make the changes to your local branch, add a new commit with those changes, and push those to your fork. GitHub will automatically update the pull request.

git add my/changed/files
git commit
git push origin my-branch

If a git conflict arises, it is necessary to synchronize your branch with other changes that have landed upstream by using git rebase:

git fetch upstream HEAD
git rebase FETCH_HEAD
git push --force-with-lease origin my-branch

Important: The git push --force-with-lease command is one of the few ways to delete history in git. It also complicates the review process, as it won't allow reviewers to get a quick glance on what changed. Before you use it, make sure you understand the risks. If in doubt, you can always ask for guidance in the pull request.

There are a number of more advanced mechanisms for managing commits using git rebase that can be used, but are beyond the scope of this guide.

Feel free to post a comment in the pull request to ping reviewers if you are awaiting an answer on something.

Approval and request changes workflow

All pull requests require "sign off" in order to land. Whenever a contributor reviews a pull request they may find specific details that they would like to see changed or fixed. These may be as simple as fixing a typo, or may involve substantive changes to the code you have written. While such requests are intended to be helpful, they may come across as abrupt or unhelpful, especially requests to change things that do not include concrete suggestions on how to change them.

Try not to be discouraged. If you feel that a particular review is unfair, say so, or contact one of the other contributors in the project and seek their input. Often such comments are the result of the reviewer having only taken a short amount of time to review and are not ill-intended. Such issues can often be resolved with a bit of patience. That said, reviewers should be expected to be helpful in their feedback, and feedback that is simply vague, dismissive, and unhelpful is likely safe to ignore.

Step 10: Landing

In order to land, a pull request needs to be reviewed and [approved][] by at least one FinetuneLLMs Collaborators. After that, as long as there are no objections from other contributors, the pull request can be merged.

When a collaborator lands your pull request, they will post a comment to the pull request page mentioning the commit(s) it landed as. GitHub might show the pull request as Closed at this point, but don't worry. If you look at the branch you raised your pull request against, you should see a commit with your name on it. Congratulations and thanks for your contribution!

Reviewing pull requests

All FinetuneLLMs contributors who choose to review and provide feedback on Pull Requests have a responsibility to both the project and the individual making the contribution. Reviews and feedback must be helpful, insightful, and geared towards improving the contribution as opposed to simply blocking it. Do not expect to be able to block a pull request from advancing simply because you say "No" without giving an explanation. Be open to having your mind changed. Be open to working with the contributor to make the pull request better.

When reviewing a pull request, the primary goals are for the codebase to improve and for the person submitting the request to succeed. Even if a pull request does not land, the submitters should come away from the experience feeling like their effort was not wasted or unappreciated. Every pull request from a new contributor is an opportunity to grow the community.

Review a bit at a time

Do not overwhelm new contributors.

It is tempting to micro-optimize and make everything about relative performance, perfect grammar, or exact style matches. Do not succumb to that temptation.

Focus first on the most significant aspects of the change:

  1. Does this change make sense for FinetuneLLMs?
  2. Does this change make FinetuneLLMs better, even if only incrementally?
  3. Are there clear bugs or larger scale issues that need attending to?
  4. Is the commit message readable and correct? If it contains a breaking change is it clear enough?

When changes are necessary, request them, do not demand them, and do not assume that the submitter already knows how to add a test or run a benchmark.

Specific performance optimization techniques, coding styles, and conventions change over time. The first impression you give to a new contributor never does.

Nits (requests for small changes that are not essential) are fine, but try to avoid stalling the pull request. Most nits can typically be fixed by the FinetuneLLMs collaborator landing the pull request but they can also be an opportunity for the contributor to learn a bit more about the project.

It is always good to clearly indicate nits when you comment: e.g. Nit: change foo() to bar(). But this is not blocking.

If your comments were addressed but were not folded automatically after new commits or if they proved to be mistaken, please, hide them by using github ui with the appropriate reason to keep the conversation flow concise and relevant.

Be aware of the person behind the code

Be aware that how you communicate requests and reviews in your feedback can have a significant impact on the success of the pull request. Yes, we may land a particular change that makes FinetuneLLMs better, but the individual might just not want to have anything to do with FinetuneLLMs ever again. The goal is not just having good code.

Respect the minimum wait time for comments

There is a minimum waiting time which we try to respect for non-trivial changes, so that people who may have important input in such a distributed project are able to respond.

For non-trivial changes, pull requests must be left open for at least 48 hours. Sometimes changes take far longer to review, or need more specialized review from subject-matter experts. When in doubt, do not rush.

Trivial changes, typically limited to small formatting changes or fixes to documentation, may be landed within the minimum 48 hour window.

Abandoned or stalled pull requests

If a pull request appears to be abandoned or stalled, it is polite to first check with the contributor to see if they intend to continue the work before checking if they would mind if you took it over (especially if it just has nits left). When doing so, it is courteous to give the original contributor credit for the work they started (either by preserving their name and email address) in the commit log, or by using an Author: meta-data tag in the commit.

Approving a change

Any FinetuneLLMs core collaborator is authorized to approve any other contributor's work. Collaborators are not permitted to approve their own pull requests.

Collaborators indicate that they have reviewed and approve of the changes in a pull request either by using GitHub's Approval Workflow, which is preferred, or by leaving an LGTM ("Looks Good To Me") comment.

When explicitly using the "Changes requested" component of the GitHub Approval Workflow, show empathy. That is, do not be rude or abrupt with your feedback and offer concrete suggestions for improvement, if possible. If you're not sure how a particular change can be improved, say so.

Most importantly, after leaving such requests, it is courteous to make yourself available later to check whether your comments have been addressed.

If you see that requested changes have been made, you can clear another collaborator's Changes requested review.

Change requests that are vague, dismissive, or unconstructive may also be dismissed if requests for greater clarification go unanswered within a reasonable period of time.

Use Changes requested to block a pull request from landing. When doing so, explain why you believe the pull request should not land along with an explanation of what may be an acceptable alternative course, if any.