Platform-releng/Git Workflows

From Eclipsepedia

Jump to: navigation, search

We'd like to capture some common CVS workflows used by the Eclipse Project and spell out the git/EGit equivalent. Please read this page even if you don't use EGit. It contains important instructions on how to setup your repository.

Please read some of Pro Git to get a feel for how git repositories work. Refer to the EGit/User Guide for more detailed instructions and pictures.

Contents

Getting EGit

You can install EGit 2.0.0 or later from [1].

If you would like to keep up with their current bug fixes, install EGit/JGit from their nightly build site http://download.eclipse.org/egit/updates-nightly.

Configure the workspace

Open the Team > Git > Configuration preference page and select the User Settings tab.

  • Add entries for user.name and user.email. If you don't want to share your e-mail you can also use your committer account ID. Note that you will not be able to push changes to the repository if the latter property is not matching with your records at the Eclipse Foundation.
  • Add entry branch.autosetuprebase = always
  • Add entry core.autocrlf = false

On the General > Workspace preference page, set New text file line delimiter to Unix.

Clone a repo

The first step is to clone the one or more repos you need to work on. The repos should not be stored directly in your workspace. Either clone them to <workspace>/git/ or to a location outside your workspace. Then use the EGit Import Projects option to import the projects.

Refer to the EGit/User Guide for more detailed instructions and pictures.

  1. Switch to the Git Repository Exploring Perspective
  2. Use Clone a Git repository Clone a git repository
  3. you can paste in your connection URL and it should do the right thing. Some URLs (not all of them contain content right now in the testing phase). The repos that have been migrated are in bold text (Some projects use gerrit code review system so their URL's may look different):
    1. ssh://userid@git.eclipse.org/gitroot/equinox/rt.equinox.bundles.git
    2. ssh://userid@git.eclipse.org/gitroot/equinox/rt.equinox.binaries.git
    3. ssh://userid@git.eclipse.org/gitroot/equinox/rt.equinox.framework.git
    4. ssh://userid@git.eclipse.org/gitroot/equinox/rt.equinox.incubator.git
    5. ssh://userid@git.eclipse.org/gitroot/equinox/rt.equinox.p2.git
    6. ssh://userid@git.eclipse.org:29418/jdt/eclipse.jdt.core.git
    7. ssh://userid@git.eclipse.org/gitroot/jdt/eclipse.jdt.core.binaries.git
    8. ssh://userid@git.eclipse.org/gitroot/jdt/eclipse.jdt.debug.git
    9. ssh://userid@git.eclipse.org/gitroot/jdt/eclipse.jdt.git
    10. ssh://userid@git.eclipse.org/gitroot/jdt/eclipse.jdt.ui.git
    11. ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.build.git
    12. ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.git
    13. ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.incubator.git
    14. ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.ui.git
    15. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.debug.git
    16. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git
    17. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git
    18. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git
    19. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.eclipsebuilder.git
    20. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.resources.git
    21. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.runtime.git
    22. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.team.git
    23. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.text.git
    24. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.ua.git
    25. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.ui.git 
    26. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.swt.git 
    27. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.swt.binaries.git
    28. ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.common.git
  4. Next
  5. Select all branches to clone
  6. Next
  7. Confirm the location that it will clone the repository into.
  8. an initial branch of master and a remote name of origin are standard.
  9. Finish - Now just sit back while git copies the entire repo to your harddrive :-)

Configuring the repo

Make sure that you set core.autocrlf = false and on Windows core.filemode = false. The former should already be configured in the workspace, but it doesn't hurt to be explicit.

Unless you are working on topic branches, we work in a fairly linear history. Please make sure branch.branchname.rebase = true is set. If you've set branch.autosetuprebase = always as explained in #Configure_the_workspace, then this is done automatically.

Otherwise, once you've cloned a repository, you can go to the Preferences > Team > Git > Configuration page. Select your repository, select the branch you picked when you cloned the repository, and click New Entry.... Append "rebase" to the text in the 'Key' field and enter "true" as value.

RepositoryConfigurationSettings.png

Importing the projects

We need to get the projects from the repo into our workspace:

  1. right click on your newly cloned repo and select Import Projects
  2. you want Import existing projects from the Working Directory
  3. Next
  4. Select the projects that you want to import from the repository
  5. Finish

Now you can start working.

A note on deleting projects

Typically you will only want to have a subset of the projects from a given repository in your workspace. When you are no longer interested in a project, you can delete it from your workspace. However, NEVER select 'Delete project contents on disk' for a project in a git repository. If you do, Git will consider this an outgoing deletion to be committed to the remote branch. Later while working on a completely unrelated project you may accidentally commit this deletion (and you wouldn't be the first to do so).

Start working in HEAD

To start working in HEAD you must clone your repository and checkout a working copy. By default, cloning the repo checks out the master branch, which is the same as HEAD in CVS.

See #Clone_a_repo.

Refer to the EGit/User Guide for more detailed instructions and pictures.

The constant HEAD is used in GIT as well, but has a completely different meaning. In GIT HEAD means the pointer to the latest commit in your currently checked-out branch (more or less).

Start working in a new branch

For example, create the R3_7_maintenance branch for your repo. This example is for the case that your branch doesn't already exist as refs/remotes/origin/R3_7_maintenance.

Refer to the EGit/User Guide for more detailed instructions and pictures.

  1. right click on one of your projects and choose Team>Switch To>New Branch
  2. you need to pick a source ref.
    1. HEAD == current checked out commit
    2. refs/heads/master means your master branch.
    3. refs/remotes/origin/R3_7_maintenance - the existing remote branch. If you pick this one and name your local branch the same, EGit will automatically create a tracked branch.
    4. refs/tags/R3_7 is the tags to branch from
  3. name the branch R3_7_maintenance
  4. select the Rebase merge option
  5. leave "Check out the new branch" selected.

This will create a new branch for you to work on. Once you've made your initial commits, you need #Commit_changes_to_the_main_repo. Pushing up to the repo will push any new branches you've created as well.

Create a patch

We have 2 options for accepting contributions from the community. See Development Resources/Handling Git Contributions (prefered) and Git#IP_process_implications_of_DVCS. Refer to the EGit/User Guide for more detailed instructions and pictures.

To create a patch:

  1. You need to show the commits in the history view.
    1. Either right-click on a file you just committed and Show in history
    2. or right-click on the project and Show In>History, then find the commit you want
  2. right click on the commit in the History view, and select Create Patch
  3. use Next if you'd like the patch in the standard git mbox format

Apply a patch

A normal workspace patch will apply in the same fashion it does for CVS. Simply copy the file to the clipboard and paste it into the Package Explorer or use Team>Apply Patch.

Refer to the EGit/User Guide for more detailed instructions and pictures.

Patches created with git/EGit have a different pattern. They are diff statements that look like:

 diff --git a/bundles/org.eclipse.some/src/org/eclipse/some/File.java b/bundles/org.eclipse.some/src/org/eclipse/some/File.java
index 99d339f..37bcf68 100644
--- a/bundles/org.eclipse.some/src/org/eclipse/some/File.java
+++ b/bundles/org.eclipse.some/src/org/eclipse/some/File.java

To apply a patch like this, you should:

  1. copy and paste the patch into the Package Explorer
  2. Select Apply patch to the workspace root
  3. Next
  4. under Patch Options set Ignore leading path name segments to 2
  5. now you can examine the patch and apply is normally.

Gerrit workflow for a committer

Some suggested workflows for a committer working with Gerrit.

Setting up your repo

Committers usually have existing repos, and they can be modified to work wit Gerrit. First you have to go to your settings in Gerrit and select your "writable" method, HTTPS (add an HTTPS password) or SSH (upload the appropriate public key, it doesn't access your regular dev SSH keys). https://git.eclipse.org/r/#/settings/

Then you need to add another remote to your git repo. I have ssh://git.eclipse.org/platform/eclipse.platform.ui.git as one of my repos, and that maps to either:

https://pwebster@git.eclipse.org/r/platform/eclipse.platform.ui
ssh://pwebster@git.eclipse.org:29418/platform/eclipse.platform.ui

You can add these 2 properties from the command line or from EGit - Preferences>Team>Git>Configuration in the Repository Settings tab (pick the correct repo from the combo box).

git config remote.review.url ssh://pwebster@git.eclipse.org:29418/platform/eclipse.platform.ui
git config remote.review.push HEAD:refs/for/master

If you use EGit, when creating a commit for Gerrit you will need to add a Change-Id (a button in the Git Staging view). If you are using the command line, you'll need to install the commit-msg hook (Gerrit#Install_the_commit-msg_hook_in_your_repository). The commit-msg hook will add the Change-Id to all of your commits, which will have no effect if you don't push your commit to Gerrit.

Pushing a commit to Gerrit for review

A commit for review is the same as a regular commit except it has a Change-Id in the footer, and is pushed to a special refspec - HEAD:refs/for/master

Once you have a commit ready, in EGit add the Change-Id placeholder to the bottom of the commit message using the Change-Id button. From the command line make sure the Change-Id commit message hook is set up. Then commit. A unique Change-Id footer is created. If you've already committed you can just amend the commit to add the Change-Id.

Then you push the commit to Gerrit. In EGit right click on your repo in the Git REpositories view and select Push to Gerrit.... Select your review remote URI and fill in the Gerrit branch next to the refs/for combo: master, R4_2_maintenance, etc.

If you are pushing a new commit for review, the popup status dialog will display the URL to the new change that was created.

From the command line, you can use:

git push review

Updating a change during the review cycle.

You can update the patch set (commit) for a change for a number of reasons. Reviewed and you want to make the changes suggested, rebase on top of your branch tip to keep the change fresh, etc. Any commit that contains an existing Change-Id and is pushed to refs/for/<branch> will replace the existing patch set on that change.

Updating a Gerrit change that was pushed with no Change-Id

If you accidentally push to refs/for/master with no Change-Id, Gerrit will create a change for you and it will have a Change-Id. The commit message will not contain it, however.

If you need to push up a second patch set, just include the Change-Id footer with the ID from the Gerrit change webpage, and Gerrit will correctly add the new command as a patch set on the existing change.

Contributing to a build

Daily development occurs in the master branch, but weekly integration builds are done out of a branch called integration. The builder takes care of tagging the build inputs and updating bundle qualifiers for those bundles that have changed. To contribute a repository to the build, do the following:

  1. Make sure you do a fetch on your repository
  2. Ensure all desired changes for the build are in branch master
  3. In the Git Repositories view, expand Branches > Local, and select integration. Right click and select checkout. (Note: the very first time, you will need to first "create a branch" for integration.)
  4. Select branch master and select merge
  5. Right-click on the repository and select Push to Upstream
  6. Remember to switch back to master to continue more new development.

To contribute to a build from the git command line, first ensure your local master stream contains the contents you want to contribute to the build. Then, do the following:

 git pull
 git checkout integration
 git merge master
 git push origin integration

This will cause a fast-forward merge, so that both master and integration refer to the same commit. Past commits that were involved in builds will be identifiable by the build tag on that commit.

Changing what branch is built

The builder automatically takes a branch of each repository and contributes the latest branch contents to the build. The branch used for each repository is configured in the following file:

http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/tree/org.eclipse.releng/tagging/repositories.txt

To build a different branch, edit this file and enter the branch name to be built on the line next to your repository. Commit and push the change. Use branch master for current development stream builds, and Rx_y_maintenance for maintenance builds.

Creating an integration branch

If your repository does not yet have a branch called integration, it will need to be created and pushed. This is done as follows:

  1. Open the Git Repositories view, and expand Branches > Local.
  2. Right-click on master, and select Create Branch...
  3. For the branch name, enter integration. Leave the pull strategy as "None".
  4. Click Finish
  5. Right-click on the repository, and select Push...
  6. In the Push dialog, ensure the correct remote is selected and click Next
  7. In the Source ref field, select integration in the drop-down list. The destination spec will be filled in automatically. Click Add Spec to include this specification in the push operation.
  8. Click Finish to complete the push operation

For command line users, do the following:

  git checkout master
  git branch integration
  git push origin integration

Manual tagging

Tagging for weekly integration builds is not needed, but sometimes you still need to tag. For example, tagging with a Rx_y tag when a release is done. To create a tag, use git tag mynewtag from the command line or the EGit Tag Dialog. Once a tag has been added, you must push the tag to the repository.

To push a specific tag, use git push origin mynewtag from the command line or use the following steps in EGit:

  1. Team > Remote > Push...
  2. Click Next to get to the Specifications Page
  3. In the Source ref box, enter in refs/tags/mynewtag (content assist is available to quickly find a tag)
  4. Click Add Spec
  5. Click Next
  6. Assuming the dry run is successful, click Finish

You can push all tags using git push --tags from the command line or pressing the Add All Tags Spec on the Specifications Page in EGit. However, this is risky as you will push all local tags and you will replace any tags that have been deleted from the remote repository.

The e4 Git page has some helpful scripts and additional information on tagging

Branches in our Platform Repos

We use a pre-receive hook in our repositories to enforce our branching/deleting policy.

  • committers can create, push anything, and delete their own topic branches on the remote repositories. Topic branches must have the form committerId/branchName. For example, pwebster/bug372119
  • committers can only push Fast Forward merges to the other branches, like origin/master or origin/R3_7_maintenance, or someone else's topic branch
  • committers cannot create or delete non-topic branches
  • committers can only delete tags with the same pattern as topic branches, committerId/tagName

Functionality can be re-enabled by setting certain config properties on the remote repository:

  • hooks.allownonffpush = true: Allow a non-FF push to any branch
  • hooks.allowdeletebranch = true: Allow any branch to be deleted
  • hooks.allowcreatenottopicbranch = true: Allow any branch to be created, like origin/R3_8_maintenance
  • hooks.allowdeletetag = true: Allow any tag to be deleted

To temporarily set a property, a committer with shell access executes

$ git config --add hooks.<property> true

To disable all properties, execute

$ git config --remove-section hooks

Commit changes to the main repo

Committing a change to the main repo is a 2-step process in git. In git, a commit creates a commit with the changed files in your local clone repository. A push will put that commit in our main repo. Committing and pushing are distinct operations in git.

Refer to the EGit/User Guide for more detailed instructions and pictures.

To get your changes to the main repo in EGit:

  1. Do a pull or a fetch and a merge into your working branch
  2. right-click on your project and use Team>Commit
  3. Your commit message should include the bug number you are using for your fix/work.
  4. check the files that should be included in the commit in the Files section
  5. Commit

Then you need to push your changes to make sure they're visible to everyone else

  1. right-click on your project and use Team>Push to Upstream
  2. it should provide a status dialog with the refs that were updated, or a failure if the main repo has commits that you haven't either merged or rebased on.
  3. if there's a failure, you need to #Update_to_pull_in_the_latest_changes_to_HEAD or the relevant branch.


Common commit message:

 Bug 349177 - [releng] stitch ui.workbench fork back into main
Updating some code to reflect the real change

The eGit Staging View provides a way to see all changed code in your workspace. You can select a subset of the changed files to commit.

Update to pull in the latest changes to HEAD

To make changes visible from our main repo is a 2 step process in git:

  1. fetch which updates your cloned repo with all of the objects and remote branches from the main repo.
  2. merge which updates your local branch to point to the correct commit
    1. the most common case is called the "fast forward merge". That's where your repo has no local changes, and git can simply update your local master to the commit pointed to by origin/master
    2. if you have a local commit that has not yet been pushed, you might have to deal with merge conflicts:
      1. you will either have to merge the new origin/master into your master (which will lead to a merge commit with 2 parent commits)
      2. or rebase your local commit onto that last commit coming from origin/master. This leaves the history of the main repo as a simple line, and is prefered ... I think.

Refer to the EGit/User Guide for more detailed instructions and pictures.

Git can do both fetch and merge for the current branch at once:

  1. right click on your project and select Team>Pull

Making the same change in two branches

The easiest way to make the same change in two branches is by using the git "cherry-pick" command. The workflow is as follows:

  1. Make the change and commit in branch 1
  2. Check git log and copy the hash of the commit you just made to clipboard/etc
  3. Checkout branch 2
  4. Run "git cherry-pick <commit>" using the commit hash from step 2
  5. If you run git log, you will now see a new commit in branch 2 with the same contents and author as the commit on branch 1
  6. Push your changes in both branches

Merge conflicting changes

Refer to the EGit/User Guide for more detailed instructions and pictures.

If you pull changes for your branch and there are conflicts, you must merge them before you can commit your changes.

By default EGit merges conflicts into the local files using the old merge syntax Text Merge Markers. To switch to a compare editor merge, use the Team>Merge Tool menu item. See EGit/User Guide#Resolving_a_merge_conflict

Dealing with line terminators

Git has various options for normalizing line terminators between Windows (which uses CR/LF), and other platforms that use only LF. After trying various approaches, the Eclipse project has settled on the following approach:

  • Set core.autocrlf=false.
  • In your workspaces set General > Workspace > 'New text file line delimiter' to 'Unix'.
  • In the current dev branches (do not do this in maintenance branches) set 'New text file line delimiter' to 'Unix' on the 'Resource' property page of your projects.
  • If you don't work with EGit then it is your responsibility to avoid committing wrong line delimiters to the repository.
  • If using Windows, set core.fileMode=false. If you use EGit to clone the repository then this is done automatically for you.

References:

Cross-referencing between bugzilla and git

Some committers like to attach patches to bugzilla for all changes, so they have a record of exactly what changes occurred to fix the bug. With git there is a better way:

  • Push the change to git.eclipse.org
  • Navigate to git.eclipse.org in your web browser
  • Find your repository, and click on the "Log" tab
  • Click on your commit, which should be near the top of the log at this point
  • You are now on a page that shows a graphical diff of call changes that occurred in that commit. Copy the URL of this commit page into a bugzilla comment.

Now if someone in the future wants to see what changes were made to fix the bug, they have a one click link to see all the changes.

Convert .cvsignore to .gitignore

There are subtle differences in semantics between .cvsignore and .gitignore, so you shouldn't just rename these files when moving to Git. At least prepend a "/" to each line in .gitignore, to ensure a pattern doesn't match in all subfolders (unfortunately, this is not explained well in the gitignore docs).

Good practice is to create a .gitignore directly in the top-level folder of the Git repository and add /*/bin/ or /*/*/bin/ (depending on repo layout). Then, you can remove the bin entry from project-level ignore files (and also remove the ignore file if that was the only entry).

Pruning corrupt repository branches

Sometimes a branch in your repository might get corrupted and you may see errors such as "Could not read an object while parsing commit ...." in Egit or "Branch .... does not point to a valid object". The following command comes handy in such scenarios: git remote prune origin

(Always prefix a --dry-run to see what this command is going to do before you go ahead and execute it.)


References

For background reading on Git, see Git#Resources.