Jump to: navigation, search

Difference between revisions of "Development Resources/Automatic IP Log"

Line 7: Line 7:
  
 
The Automated IP Log Tool automatically generates an IP Log using information that is available to the Eclipse Foundation. The list of committers, for example is generated using information provided by the [http://www.eclipse.org/dash Dash] project which itself pulls information out of eclipse.org CVS and SVN repositories.
 
The Automated IP Log Tool automatically generates an IP Log using information that is available to the Eclipse Foundation. The list of committers, for example is generated using information provided by the [http://www.eclipse.org/dash Dash] project which itself pulls information out of eclipse.org CVS and SVN repositories.
 +
 +
{{Warning|Dash gathers its information via a long-running process that is initiated weekly (on Saturday night). If you make changes to the data that Dash uses to run, those changes will not be reflected immediately. If necessary, the scripts can be initiated manually by [mailto:emo@eclipse.org the EMO] if required (ask nicely).}}
  
 
To fully leverage the value of the Automated IP Log Tool, you need to:
 
To fully leverage the value of the Automated IP Log Tool, you need to:
Line 142: Line 144:
 
==Problems with the IP Log?==
 
==Problems with the IP Log?==
 
If you're having trouble updating the IP log or if the IP log is not showing enough information, please file a bug (use [https://bugs.eclipse.org/bugs/enter_bug.cgi?assigned_to=portal-inbox%40eclipse.org&blocked=&bug_file_loc=http%3A%2F%2F&bug_severity=normal&bug_status=NEW&comment=&component=Portal&contenttypeentry=&contenttypemethod=autodetect&contenttypeselection=text%2Fplain&data=&dependson=&description=&flag_type-1=X&flag_type-2=X&flag_type-4=X&flag_type-6=X&flag_type-7=X&flag_type-8=X&form_name=enter_bug&keywords=&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=All&priority=P3&product=Community&qa_contact=&rep_platform=All&short_desc=&version=unspecified this link]) and we'll work hard to make things right.
 
If you're having trouble updating the IP log or if the IP log is not showing enough information, please file a bug (use [https://bugs.eclipse.org/bugs/enter_bug.cgi?assigned_to=portal-inbox%40eclipse.org&blocked=&bug_file_loc=http%3A%2F%2F&bug_severity=normal&bug_status=NEW&comment=&component=Portal&contenttypeentry=&contenttypemethod=autodetect&contenttypeselection=text%2Fplain&data=&dependson=&description=&flag_type-1=X&flag_type-2=X&flag_type-4=X&flag_type-6=X&flag_type-7=X&flag_type-8=X&form_name=enter_bug&keywords=&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=All&priority=P3&product=Community&qa_contact=&rep_platform=All&short_desc=&version=unspecified this link]) and we'll work hard to make things right.
 +
 +
=Manually Generate the IP Log=
 +
 +
An XML file containing the IP Log data can be generated via a RESTful web services call to [nowiki]http://eclipse.org/projects/xml/project_ip_data.php?id=<project id>[/nowiki] (where project id is--oddly enough--the id of your project; e.g. technology.egit). You an use your XML editor of choice to make the necessary modifications.
 +
 +
Use the [http://www.eclipse.org/projects/xml/iplog.xsd iplog.xsd] file to convert the modified IP Log to HTML and submit it via email to [mailto:emo@eclipse.org,emo-ipteam@eclipse.org?subject=Request%20IP%20Log%20Review%20 the EMO and EMO IP team] with a short note requesting the review. Please include the id of your project in the subject line. If the IP Log is being submitted for a release review, please indicate the version of the release in the message text.
 +
  
 
''This page is moderated by the EMO''
 
''This page is moderated by the EMO''
 
[[Category:Development_Resources]]
 
[[Category:Development_Resources]]

Revision as of 10:19, 11 April 2013

The Foundation automatically produces IP Logs for each project (bug 220977). This page describes how the automatic IP Log is generated (by the system) and maintained (by the project team) and processed (by Eclipse Legal).

What is an IP Log?

An IP Log is a record of the IP contributions to a project. This includes such as a list of all committers, past and present, that have worked on the code and (especially) those who have made contributions to the current code base. The expected contents of a log is described on the Project IP Log page.

What is the Automated IP Log Tool?

The Automated IP Log Tool automatically generates an IP Log using information that is available to the Eclipse Foundation. The list of committers, for example is generated using information provided by the Dash project which itself pulls information out of eclipse.org CVS and SVN repositories.

Warning2.png
Dash gathers its information via a long-running process that is initiated weekly (on Saturday night). If you make changes to the data that Dash uses to run, those changes will not be reflected immediately. If necessary, the scripts can be initiated manually by the EMO if required (ask nicely).


To fully leverage the value of the Automated IP Log Tool, you need to:

Your project's metadata is used to determine the identities of the source code repositories that Dash needs to scan to find out committer information. Specifically, you need to specify, in the "Source Repositories" section, a list of paths to Git and SVN repository locations. With this information specified, not only will the Automated IP Log tool be able to build the "Committers" section of the IP Log for you, but your project's Project Summary page will also be updated to accurately reflect your project's commit activity.

The Automated IP Log tool populates the Developers section with information gathered from Git and Bugzilla. This section lists contributions from non-committers (this is time-sensitive, so contributions made by current committers before they became committers will also be included).

Git commits contributed by non-committers are identified by the author identity on the commit record; the "Author" and "Author Name" fields must be set to the identity of the actual author of the commit.

Only non-committer contributors are recorded in the generated log. To aid the tool in generating correct results, you need to mark Bugzilla attachments with the iplog+ flag. In setting this flag, the Automated IP Log tool assumes that the person who attached the bug is the contributor. To comply with the website terms of use, the person who attaches the contribution really needs to be the person who has permission to make it available. You should ensure that this is the case before including the code in your project's repository and flagging the entry.

Alternatively, you can flag an entire Bugzilla entry with +iplog. Doing so, however, indicates to the Automated IP Log tool that every single comment made by a non-committer in the bug report represents a potential contribution. For your own sanity, it's a good practice to ask contributors to provide and attach patches that can be individually marked. Marking an entire bug represents an ongoing maintenance issue as new comments added to the bug from non-committers will show up in the generated log.

Note that contributions flagged in Bugzilla will only appear in the IP Log if the bug is marked FIXED or CLOSED.

The Bugzilla Contribution Review Tool can help you identify potential contributions.

The Third-Party Software section of the log is populated from our IPZilla Bugzilla instance. The IP Team will mark your contributions in such a way that they will appear in the log. If third party software is not appearing properly, contact the EMO IP Team to make corrections.

The rest of this document is concerned with tuning the output of the Automated IP Log generator.


Using the Automated IP Log

Warning2.png
The Automated IP Log's editing functionality is broken, considered retired (though not completely removed), and due for replacement/integration with the Project Management Infrastructure. It has been our experience that the edit functionality is rarely (if ever) actually required; problems that require editing can almost always be addressed by correcting the underlying data.


To start with the Automated IP Log, you need to first navigate to the page. The URL is of the form: http://www.eclipse.org/projects/ip_log.php?projectid=[projectId] (e.g. [1]).

Note that the log found at this URL is marked "Tentative". This is automatically-generated content that is only considered official when it is actually submitted. To submit the log for review, you must log-in (using the link at the bottom of the page), correct any errors that you find (see below), and click "Submit".

Note that you can adjust the set of projects that contribute data to the log by clicking the "Adjust" ("the set of projects to match the upcoming release") button. A log can be submitted for multiple projects; you might do this if your project provides a "roll-up" release and download that includes code from one or more subprojects.

When the log is submitted, it is reviewed by the IP Team. They may have questions, or ask for clarifications. You may be asked to resubmit the log after you have made some changes. When the process is complete, the IP Team will email you a PDF that contains the official log; you should make this available for your consumers and adopters to review.

Third-Party Code

The Third-Party Code section of the IP Log shows the approved Contribution Questionnaires from ipzilla. The CQs listed in this table are those that:

  • state=approved, or
  • RESOLVED/FIXED or CLOSED/FIXED (regardless of state), and
  • contain the keyword "nonepl"

Correcting Errors

  • Missing CQ. If a CQ is missing from this list, (a) check its status, resolution, and state: make sure that they match one of the above cases, and (b) check to ensure "nonepl" keyword is included in order to identify the code as third party. If not, contact Eclipse Legal.
  • Extra CQ. If a CQ is listed but does not belong to this project, then contact Eclipse Legal to have the ipzilla database corrected.
  • Missing or Incorrect License. If a CQ is missing the license information, contact Eclipse Legal to have the correct license information added to the database.

Unused Third-Party Code

If the project has CQs that approve the use of certain third-party code, but that code is not currently being used for this release, it is listed in the separate Unused Third-Party Code table. The CQs listed in this table are those that have one of the following keywords:

  • obsolete - this third-party code was used in previous releases, but it has been superseded or is obsolete and thus is no longer being used in this release and there is no plan to use it in a future release.
  • unused - the project requested approval of this third-party code in anticipation of using this code, but at this time the project is not using this code in this release. The project reserves the right to use the code in a future release.
  • withdrawn - the project requested approval of this third-code code but subsequent to receiving approval and prior to using the code, decided not to use said code. This code has not been used in any release, is not being used in this release, and there is no plan to use it in a future release.

Correcting Errors

  • CQ Is Being Used. If a CQ is listed as unused but is actually being used, contact Eclipse Legal to have the keywords on CQ removed/changed.
  • CQ Should Be Unused. If a CQ should be in the unused list instead of one of the other lists, contact Eclipse Legal to have the appropriate keyword added.

Works With and Pre-Req Dependencies

If the project has Works With, Pre-Req, or Exempt Pre-Req Dependencies, these will be listed in a separate table. These are also Contribution Questionnaires from ipzilla. These CQs are:

  • state=exempt_prereq
  • state=prereq
  • state=workswith

Correcting Errors

  • Missing Pre-Req. If an pre-req is missing from this list, first check that to see if there is a CQ for it. If there is no CQ, submit one (using the portal). If there is a CQ, check that its state is "workswith", "prereq", or "exempt_prereq" - if not, contact Eclipse Legal.

Pending Contribution Questionnaires

If the project has any pending and not yet approved or rejected Contribution Questionnaires, these are listed in a separate table. These represent potential IP issues and thus must be considered carefully during a Release Review.

Correcting Errors

  • Unneeded CQ. If the code represented by a pending CQ is no longer needed by the project, use ipzilla to withdraw the CQ.

Committers

The Committers section of the IP Log uses two information sources to generate the two lists:

  1. The list of all committers who have committed code to any of the source repositories from the Commits Explorer. This is the basis of the "Past and Present" list. The commits explorer list is driven by the project meta data source repository item(s).
  2. The list of all past and present committers from the Foundation's internal database of committers and projects. Committers here but not in in the "Past and Present" list are put on the "Never Active" list.

Correcting Errors

If you need to make corrections to this section, then you may need to manually generate the IP Log.

  • Should Be Active. If a committer is listed as "Never Active" but should be listed in the active "Past and Present" table, then you should confirm that the project metadata correctly lists all project source code repositories. The easiest way to a missing committer is to have them make a commit (in any branch).
  • Question Mark. If a "xxx???" person is listed as a committer, that means that the unix login "xxx" committed code to the source code repository, but there is no record of "xxx" as having been a committer on the project. This is a potential IP cleanliness problem, so contact the EMO to resolve it.
  • Extra Committer. If someone is listed as a "Never Active" committer, but you know that they were never even a committer at all, then there is an incorrect record in the Foundation database - contact the EMO. On the other hand, if someone is listed as a "Past and Present" committer, but you know that they were never a committer, then something larger is awry because there are source code check-ins by that person. Use the commits explorer to determine what check-ins they made and contact the EMO if there are any errors.
  • Missing Committer. If someone who is/was a committer with a unix account and is missing from both the "Past and Present" and the "Never Active" tables, then confirm that the project metadata correctly lists all project source code repositories and contact the EMO

Contributors

The Contributors section of the IP Log uses two information sources: bugzilla (driven by the project meta data bugzilla item(s)) and Git repositories (specified by the project metadata sourcerepositories items.

For Bugzilla, the automatic IP Log considers:

  1. all people who were not committers when they attached:
    1. an attachment to a bug
    2. and the attachment is flagged with "iplog+"
    3. and is attached to a RESOLVED/FIXED, VERIFIED/FIXED, or CLOSED/FIXED bug
    For example, see bug 69671; the contribution is in a flagged attachment so it shows up in the CDT IP Log.
  2. all people who were not committers when they commented on a bug:
    1. that is flagged with "iplog+"
    2. and RESOLVED/FIXED, VERIFIED/FIXED, or CLOSED/FIXED
    For example, see bug 190154 in which the contribution is in comments #0 and #11. Then see the BIRT IP Log. Note that comments #1-10 and #12 were excluded by the project lead using the exceptions mechanism.

In plain English, this means there are two different ways to mark a contribution in bugzilla. If the contribution is an attachment to a bug such as a patch, the attachment itself should be marked with the iplog+ flag. If the contribution is in the form of a comment in the bug, then the iplog+ flag should be set on the bug itself. Since this will mark all non-committers who commented on the bug as contributors, you typically then need to use the exceptions mechanism to exclude comments that are not contributions. Typically these two mechanisms are mutually exclusive: if the contribution is an attachment, you do not also need to set iplog+ flag on the bug itself (unless the bug also has other contributions in the form of comments).

For contributions made through Git, the automatic IP Log grabs all commit records with a author who is not a project committer. For more information, please see Handling Git Contributions.

Correcting Errors

  • No Name. If the contributor record for person X shows an email address but no name, then person X has not entered a name in Bugzilla. Have X go to bugzilla Preferences and enter their correct name.
    Iplog-correcting-errors-7.png
  • Error Messages. If the tentative IP log shows errors (red error messages) in product name or component name, then the team should check the Bugzilla entries in the project metadata - the product name and (optional) component names must be an exact match with Bugzilla, capitalization and all.
  • Extra Contributor. If a person (person X) is listed as a Contributor but the team knows that their patch was never used/is not currently in use, there are three solutions: (1; preferred) remove the "iplog+" flag from the bugzilla attachment.
  • Missing Contributor. If person X should be listed as a Contributor but is not, then be sure that person X has attached a patch to a fixed bug and that patch is flagged with "iplog+". All external code contributions should come through bugzilla patches, so if a code contribution first comes through email, the contribution must also add a patch. Note that person X must add the patch because if person Y adds the patch for person X, then person Y will be listed as the contributor.
    Iplog-correcting-errors-6.png
  • Committer is Listed. If committer X is listed as a Contributor, check the following:
    1. First, a committer can be listed as a contributor if he or she was a contributor, contributed some code, and then later became a committer. He/she will be listed as a contributor for the patches he/she submitted before becoming a committer.
    2. Check that person X's email address in bugzilla matches any one of their email addresses on file with the Foundation. Have person X use the the portal to check to verify their email addresses. If their bugzilla address does not match the Foundation committer records, we have no way to associate their bugzilla records with their committer status.

Keeping the IP Log Current

Keeping the automatic IP log current is fairly simple:

  1. Make sure that all third-party code and dependencies are filed as CQs. You can use this script to view all content in your download directory organized by CQ. Anything showing up red on this scan may need a CQ.
  2. If a non-committer patch is accepted, mark it with the iplog+ flag.
  3. If a non-committer bug without a patch is accepted, mark the whole bug with the iplog+ flag and then go to the automatic IP log to exclude the non-contribution comments.
  4. Use the portal to deactivate inactive committers

Choosing the Set of Projects

The simplest case is an IP Log for a single project. However, many project teams work together to create a unified release across a set of projects. In this case, the IP Log must also be an aggregated/unified IP Log across the same set of projects. At the bottom left of the IP Log page is a "adjust the set of projects" link. That link leads to the selector page. Use the checkboxes to select a set of projects and generate a unified IP Log for all of those projects.

Iplog-correcting-errors-8.png

Submitting the IP Log to Eclipse Legal

Single or unified-multi-project, either way the IP Log can be submitted to Eclipse Legal for review using the link at bottom center of the page. Click that, fill out the reason, and voila, the a frozen copy of the log is sent to Legal.

Iplog-correcting-errors-9.png

You must send your IP log to Eclipse Legal at least three weeks in advance of your scheduled Release Review to guarantee your review date. Sending your IP log in late will jeopardize your review date due to Eclipse Legal's workload.

Problems with the IP Log?

If you're having trouble updating the IP log or if the IP log is not showing enough information, please file a bug (use this link) and we'll work hard to make things right.

Manually Generate the IP Log

An XML file containing the IP Log data can be generated via a RESTful web services call to [nowiki]http://eclipse.org/projects/xml/project_ip_data.php?id=<project id>[/nowiki] (where project id is--oddly enough--the id of your project; e.g. technology.egit). You an use your XML editor of choice to make the necessary modifications.

Use the iplog.xsd file to convert the modified IP Log to HTML and submit it via email to the EMO and EMO IP team with a short note requesting the review. Please include the id of your project in the subject line. If the IP Log is being submitted for a release review, please indicate the version of the release in the message text.


This page is moderated by the EMO