Difference between revisions of "EclipseLink/Development/Process/Git"

Revision as of 13:48, 3 October 2012

EclipseLink Development in Git

This page is for the Git usage portion of the dev process. It does not discuss issues with the build in Git, for more information on that please see: wiki.eclipse.org/EclipseLink/Build/Git .

EclipseLink Git Repo

The EclipseLink runtime repository is located here:

http://git.eclipse.org/gitroot/eclipselink/eclipselink.runtime.git The steps to connect to the repo can be found after the overview section below.

If you wish to connect to any of the other EclipseLink git repos (a full list can be found here: http://git.eclipse.org/c/eclipselink) please follow the same instructions for each repo.

A useful url to browse history of the git repo is: http://git.eclipse.org/c/eclipselink/eclipselink.runtime.git

Brief Overview

Git is a Distributed Version Control System (DVCS), which means there is no single point-of-failure and one can do useful work without a server. The ability to work while disconnected is very useful if a server is down or if the network connection to a server is unreliable, slow or firewalled.

The distributed nature of Git means that the source code is inherently backed-up across all the various 'clones' that may exist 'out there.' In addition, Git supports types of work-flows that are different from those supported by Subversion; these work-flows, while unfamiliar, are quite powerful and can 'overlap' - one developer may prefer a 'golden repository' work-flow while another likes 'trusted-lieutenants'; both can be supported simultaneously by the same repository.

How does Git differ from SVN?

Beyond the obvious distributed vs. central repository concept, one of the great strengths of Git is its on-disk representation of a repository. When you 'checkout' from SVN, you get a working-directory tree of files and folders that represent the 'tip' of that particular SVN branch. If you need to do some operation, say examine the commit history for some file, you must go back to the server to get the metadata for those commits. When you 'clone' a Git repository you (also) get a working-directory tree of files and folders but also the complete commit history for the entire repository. Operations to compare commits or revert the working-directory to a previous state all take place a the speed of the local filesystem without any network round-trips. Further, the history for the repository is stored in an efficient 'packed' representation.

How do patches in Git differ from in SVN?

A normal patch identifies the artifacts that have changed, typically using some directory-path+filename to specify where to apply additions, modifications or deletions. Git patches work mostly the same way, but in addition to path information, each commit (or patch) contains the cryptographic signature (SHA-1) of the parent (or parents) commit that links to it:

Basically, a Git commit (patch) does not live in isolation - it is part of a tree-structure and links to the history of the repository.

How do I get started?

The 'core' Git distribution can be downloaded from the Git download site (http://git-scm.com/download) - there are links for a variety of operating systems (Linux, Mac OS X, Windows).

Git on Windows

For those using Windows (XP or 7), TortoiseGit is available.

• pre-requisite: install Windows command-line msysgit Git tools first "Full installer for official Git for Windows"
  • Install Wizard (accept most defaults)
  • Command Line: Use Git Bash Only
  • Choosing the SSH executable: Use TortoisePLink (comes from Putty, integrates well with Windows)
• Tortoise Git Install Wizard
  • asks about SSH: use same answer as above

Accessing the EclipseLink Git repository

Read-only access through HTTP

Read-only access to the EclipseLink Git repository can be achieved through HTTP.

$ git clone http://git.eclipse.org/gitroot/eclipselink/eclipselink.runtime.git

Committer access through SSH

Step 1: Git Committer Identity

When a commit is made in Git, the commit has metadata identifying two things:

1. the author (name and email): who created the change, and
2. the committer (name and email): who committed the change to the repository

(of course for many commits the author IS the committer so only the author is identified)

The Eclipse Foundation uses these fields as part of its IP process - only committers to a project can change source stored on a Foundation's server. However, a committer may make changes on behalf of others - this enables collaboration with parties that have not gone through the Eclipse IP due diligence process. This especially is useful if say the third-party just wanted to contribute a few one-of patches: the administrative overhead of the Eclipse IP due diligence process would likely 'scare-off' most contributions (for more information, please see Handling Git Contributions)

First setup your ~/.gitconfig file:

[user]
        # email address linked to my EclipseLink committer id minorman
        email = michael.norman@oracle.com
        name = Mike Norman

NB. Windows often has difficulty with 'dot-files' in its 'home' directory - you may have to create this file from the command-line. In addition, if your 'home' directory is on a UNC fileshare directory, the msysgit Windows-version of git tools may not be able to read or write it. I solved this issue by redefining two Windows environment variables (HOME and HOMEDRIVE):

Step 2: SSH identity

As mentioned above, most Git servers are set up so that HTTP access is read-only - in order to be able to commit, one must connect to the Eclipse Foundation Git servers over SSH:

Creating an SSH identity

You need to generate an SSH public/private key-pair from the command prompt:

prompt > ssh-keygen -t rsa -C "your_eclipse_committer_id@dev.eclipse.org" -f committerid
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in commiterid
Your public key has been saved in commiterid.pub.
prompt > ls committerid*
commiterid             commiterid.pub

The file ending in .pub is the public portion of the key-pair. SFTP the public key, or send an email message asking 'webmaster@eclipse.org' to place it in the appropriate place in your home directory on the Eclipse Foundation's server. The private portion must be moved to your local home directory ~/.ssh/committer.ppk. Now set permissions on your private key:

prompt > chmod 700 ~/.ssh
prompt > chmod 600 ~/.ssh/committer.ppk

Instead of emailing 'webmaster@eclipse.org' you can upload the key yourself:

sftp dev.eclipse.org
cd /home/data/users/<your_user_name>/.ssh/
put committer.pub
mv committer.pub authorized_keys
chmod 700 authorized_keys

Creating an SSH identify on Windows

PuttyGen is an alternative to "ssh-keygen" for generating a key pair. (In general, ssh-keygen is preferable as the final step with puttygen involves cut/pasting the public key, which is more error prone)

• run PUTTYGEN.EXE - make sure that SSH-2 RSA/1024 is selected
• Press 'Generate'
• In the 'Key comment' field, replace the entry starting with 'rsa-key ...' with your_eclipse_committer_id@dev.eclipse.org
• Save the private key to %HOMEDIR%\.ssh\committer.ppk
• Save the text from the box "Public key" into a new file: %HOMEDIR%\.ssh/authorized_keys

The last step is necessary because PUTTYGEN has a custom format for the public portion of the key-pair that will not work when uploaded to the Eclipse Foundation's server. You must save the text from the box 'for pasting' - the key is in the same format as generated by ssh-keygen

Getting through a firewall

In your home ~/.ssh/ directory, you must create a config file

prompt > cd ~/.ssh
prompt > touch config
prompt > chmod 600 config

The documentation for the ~/.ssh/config file can be found here. The particular features to focus upon are:

• the ability to specify a particular identity file, and
• the ability to specify a command to be run whenever we attempt to connect to a specific host:

Host git.eclipse.org
	Hostname git.eclipse.org
	User committer
 	IdentityFile ~/.ssh/committer.ppk
	ProxyCommand /c/windows/connect.exe -H firewall_host:firewall_port %h %p
or 
	ProxyCommand /usr/local/bin/corkscrew firewall_host firewall_port %h %p

The Windows executable connect.exe file can be found here; the corkscrew package (which supports many Unix-style operating systems) can be found here

Cloning the EclipseLink repositories

You should now be able to use SSH to clone the EclipseLink Git repositories

prompt > git clone ssh://committerid@git.eclipse.org/gitroot/eclipselink/eclipselink.runtime.git

Best practices for starting EclipseLink development with Git

This section highlights a simple, basic approach that we recommend for developing EclipseLink on Git, along with the most often used commands.

Creating a patch

To create a patch using TortiseGit using the "Create Patch Serial" option in the menu your modified filed must first commit your files. If you wish to create a patch for review before committing those files you can select "Check for Modifications" and on that window there is a "Save Unified Diff" button that will let you create a patch file.

Notes on push and pull

By default, performing git pull pulls in changes from origin to your current branch. git push, however, will push changes from ALL matching branches. One issue here is that you may have committed changes in a given branch that aren't ready to be pushed. Another is that you will need to pull on all branches that have had commits pushed to origin by other committers before being able to push. One way around this is to set the default push value to 'current' in your .gitconfig. This can be done by executing the following: git config --global push.default current. This will tell Git to only push the branch you are currently in, allowing you to execute git push without any other options. You can also use git push origin master to push changes on the master branch to origin, or git push origin HEAD to push the current branch to the same name on the remote.

Backing out a commit

If you have performed a commit and realize that something is wrong and you need to make changes, then re-commit, you can do the following: git reset HEAD~1. This will back out the commit, leaving the files ready for more edits, etc. then a re-commit. To permanently remove a commit, i.e. destroy it, use git reset --hard HEAD~1. To move back two commits use git reset HEAD~2, and so on.

What do I do when I am working on a large task, and have to put it aside to complete a shorter task?

Use git stash A more detailed example of a workflow using stash can be found here

What is the difference between a git fetch, git pull and git clone?

• git clone: creates a complete copy of a remote git repository with all the branches and commit history of the original
• git pull: updates a local git repository from a remote repository with all committed data and merges everything into the local working directory (i.e. could have merge conflicts)
• git fetch: same as git pull but doesn't merge into the working directory (i.e. postpone any merge conflicts)

Workflow overview

1. working directory: regular directory where work is done
2. staging area: 'special' directory/cache with potential commits
3. repository: committed data

View Online

How is merging different between Git and SVN

One fundamental difference between Git and SVN is branching - a branch in SVN is a very expensive artifact while in Git it is extremely lightweight - the additional cost to a local repository clone or a remote server is negligible. This leads to workflows where one creates a new branch for each feature or each refactoring or even each bug. When you are done, it is easy to merge changes from a branch into 'master' (like SVN 'trunk'). Merging can be done either manually or through tools (e.g. Beyond Compare) that can be integrated into the Git merge process.

Simple Workflow

prompt > svn checkout: http://somewhere.com/repository/proj/trunk

prompt > git clone git://somewhere.com/proj.git

Add a file to be tracked for version control

prompt > svn add path/to/file
prompt > svn status

prompt > git add path/to/file
prompt > git status

  Commit

prompt > svn commit –m {message}
NB – SVN commit only works to remote server

prompt > git commit –m {message}
NB – git commit is local only

Fixing mistakes
Restore (revert to) a pristine version:

prompt > git reset ––hard HEAD

Or further back: as if previous 2 commits never happened

prompt > git reset HEAD^^ (or HEAD~2)

Back to the very beginning:

prompt > git reset ORIG_HEAD

Send committed data to remote repository

prompt > git push origin
NB – 'origin' is default name of remote’s master branch

Get new committed data from remote repository

prompt > svn update

prompt > git pull

Other Workflows

Central Repository Workflow
A very common workflow, especially from people transitioning from a centralized system (SVN). Git will not allow you to push if someone has pushed since the last time you fetched, so a centralized model where all developers push to the same server works just fine.

Integration Manager Workflow
Another common workflow is where there is an integration manager — a single person who commits to the 'golden' repository, and then a number of developers who clone from that repository, push to their own independent repositories and ask the integrator to pull in their changes. This is the type of development model you often see with open source or GitHub repositories.

Trusted Lieutenants Workflow
For more massive projects, you can setup your developers similar to the way the Linux kernel is run, where people are in charge of a specific subsystem of the project ('lieutenants') and merge in all changes that have to do with that subsystem. Then another integrator (the 'dictator') can pull changes from only his/her lieutenants and the push to the 'blessed' repository that everyone then clones from again.