Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Development Resources/Automatic IP Log"

m (What is the Automated IP Log Tool?)
(16 intermediate revisions by 3 users not shown)
Line 1: Line 1:
The Foundation automatically produces IP logs for each project ({{bug|220977}}). This page describes how the automatic IP Log is generated (by the system) and maintained (by the project team) and processed (by Eclipse Legal).
+
The Foundation automatically produces [[Development Resources/IP Log|IP Logs]] for each project ({{bug|220977}}). This page describes how the automatic IP Log is generated (by the system) and maintained (by the project team) and processed (by Eclipse Legal).
  
 
=What is an IP Log?=
 
=What is an IP Log?=
An IP Log is a record of the IP contributions to a project. This includes such as a list of all committers, past and present, that have worked on the code and (especially) those who have made contributions to the current code base. The expected contents of a log is described on the [http://www.eclipse.org/projects/dev_process/project-log.php Project IP Log] page.
+
An [[Development Resources/IP Log|IP Log]] is a record of the intellectual property (IP) contributions to a project. This includes such as a list of all committers, past and present, that have worked on the code and (especially) those who have made contributions to the current code base. The expected contents of a log is described on the [http://www.eclipse.org/projects/dev_process/project-log.php Project IP Log] page.
  
 
=What is the Automated IP Log Tool?=
 
=What is the Automated IP Log Tool?=
  
The Automated IP Log Tool automatically generates an IP Log using information that is available to the Eclipse Foundation. The list of committers, for example is generated using information provided by the [http://www.eclipse.org/dash Dash] project which itself pulls information out of eclipse.org CVS and SVN repositories.
+
The Automated IP Log Tool automatically generates an IP Log using information that is available to the Eclipse Foundation. The list of committers, for example is generated using information provided by the [http://www.eclipse.org/dash Dash] project which itself pulls information out of source code repositories.
 +
 
 +
[[Image:IPLogGenerator.png|center]]
 +
 
 +
The IP Log generator pulls information from multiple location to assemble the log:
 +
 
 +
* Third-party libraries used by the project come from IPZilla
 +
* The Dash process scans the project source code repositories to assess committer activity
 +
* Dash also scans Git repositories for contributions
 +
** If you follow the guidelines for [[Development Resources/Handling Git Contributions|handling Git contributions]], contributions received via Git in any branch will automatically appear in the log
 +
** Dash determines repository ownership from the [[Project Management Infrastructure/Project Metadata#Source Repositories|project metadata]]; this same information is used to populate the list of repositories in the log
 +
* Contributions received as patches in Bugzilla that are marked iplog+ will automatically appear in the log
 +
** Contributions flagged in Bugzilla will only appear in the IP Log if the bug is marked FIXED or CLOSED.
 +
* License information is obtained from the Foundation database
 +
 
 +
{{Warning|Dash gathers its information via a long-running process that is initiated weekly (on Saturday night). If you make changes to the data that Dash uses to run, those changes will not be reflected immediately. If necessary, the scripts can be initiated manually by [mailto:emo@eclipse.org the EMO] if required (ask nicely).}}
  
 
To fully leverage the value of the Automated IP Log Tool, you need to:
 
To fully leverage the value of the Automated IP Log Tool, you need to:
*Keep your project metadata up-to-date
+
*Keep your [[Project Management Infrastructure/Project Metadata|project metadata]] up-to-date
 
*Mark IP Contributions in Bugzilla
 
*Mark IP Contributions in Bugzilla
*Create contribution questionnaires (CQs) where appropriate
+
*Follow the guidelines for [[Development Resources/Handling Git Contributions|handling Git contributions]]
 +
*Create [[Development Resources/Contribution Questionnaire|contribution questionnaires]] (CQs) where appropriate
  
Your project's metadata is used to determine the identities of the source code repositories that Dash needs to scan to find out committer information. Specifically, you need to specify, in the "source repository" section, a list of paths to CVS and SVN repository locations. With this information specified, not only will the Automated IP Log tool be able to build the "Committers" section of the IP Log for you, but your project's ''Project Summary'' page will also be updated to accurately reflect your project's commit activity.
+
{{Warning|Contributions should be recorded in one of Git or Bugzilla. The IP Log generator is not smart enough to detect duplicate entries.}}
  
The Automated IP Log tool populates the "Developers" section with information gathered from Bugzilla. This section lists contributions from non-committers. To aid the tool in generating correct results, you need to mark Bugzilla attachments with the +iplog flag. In setting this flag, the Automated IP Log tool assumes that the person who attached the bug is the contributor. Only non-committer contributors are recorded in the generated log. To comply with the website terms of use, the person who attaches the contribution really needs to be the person who has permission to make it available. You should ensure that this is the case before including the code in your project's repository and flagging the entry.
+
Your project's metadata is used to determine the identities of the source code repositories that Dash needs to scan to find out committer information. Specifically, you need to specify, in the "Source Repositories" section, a list of paths to source code repository locations. With this information specified, not only will the Automated IP Log tool be able to build the "Committers" section of the IP Log for you, but your project's ''Project Summary'' page will also be updated to accurately reflect your project's commit activity.
  
Alternatively, you can flag an entire Bugzilla entry with +iplog. Doing so, however, indicates to the Automated IP Log tool that every single comment made by a non-committer in the bug report represents a potential contribution. For your own sanity, it's a good practice to ask contributors to provide and attach patches that can be individually marked. Marking an entire bug represents an ongoing maintenance issue as new comments added to the bug from non-committers will show up in the generated log (bogus entries will have to be excluded - see below).
+
The Automated IP Log tool populates the '''Developers section''' with information gathered from Git and Bugzilla. This section lists contributions from non-committers (this is time-sensitive, so contributions made by current committers before they became committers will also be included).  
  
The Third-Party Software section of the log is populated from our ''IPZilla'' Bugzilla instance. The IP Team will mark your contributions in such a way that they will appear in the log. If third party software is not appearing properly, contact the [mailto:emo-ip-team@eclipse.org EMO IP Team] to make corrections.
+
Git commits contributed by non-committers are identified by the author identity on the commit record; the "Author" and "Author Name" fields must be set to the identity of the actual author of the commit.
  
The rest of this document is concerned with tuning the output of the Automated IP Log generator.
+
Only non-committer contributors are recorded in the generated log. To aid the tool in generating correct results, you need to mark Bugzilla attachments with the iplog+ flag. In setting this flag, the Automated IP Log tool assumes that the person who attached the bug is the contributor. To comply with the website terms of use, the person who attaches the contribution really needs to be the person who has permission to make it available. You should ensure that this is the case before including the code in your project's repository and flagging the entry.
  
''As of June 2009, we have started work on a new generation of tools that will be ready in the Fall of 2009. These tools will use the mechanisms indicated above.''
+
Alternatively, you can flag an entire Bugzilla entry with +iplog. Doing so, however, indicates to the Automated IP Log tool that every single comment made by a non-committer in the bug report represents a potential contribution. For your own sanity, it's a good practice to ask contributors to provide and attach patches that can be individually marked. Marking an entire bug represents an ongoing maintenance issue as new comments added to the bug from non-committers will show up in the generated log.
 +
 
 +
''Note that contributions flagged in Bugzilla will only appear in the IP Log if the bug is marked FIXED or CLOSED.''
 +
 
 +
The [http://www.eclipse.org/projects/tools/ip_contribution_review.php Bugzilla Contribution Review Tool] can help you identify potential contributions.
 +
 
 +
The '''Third-Party Software''' section of the log is populated from our ''IPZilla'' Bugzilla instance. The IP Team will mark your contributions in such a way that they will appear in the log. If third party software is not appearing properly, contact the [mailto:emo-ip-team@eclipse.org EMO IP Team] to make corrections.
  
 
=Using the Automated IP Log=
 
=Using the Automated IP Log=
  
To start with the Automated IP Log, you need to first navigate to the page. The URL is of the form: <nowiki>http://www.eclipse.org/projects/ip_log.php?projectid=[projectId]</nowiki> (e.g. http://www.eclipse.org/projects/ip_log.php?projectid=dsdp.rtsc). A link to the IP Log is provided in the project summary page for each project.
+
{{Warning|The Automated IP Log's editing functionality is broken, considered retired (though not completely removed), and due for replacement/integration with the [[Project Management Infrastructure]]. It has been our experience that the edit functionality is rarely (if ever) actually required; problems that require editing can almost always be addressed by correcting the underlying data.}}
 +
 
 +
To start with the Automated IP Log, you need to first navigate to the page. The URL is of the form: <nowiki>http://www.eclipse.org/projects/ip_log.php?projectid=[projectId]</nowiki> (e.g. [http://www.eclipse.org/projects/ip_log.php?projectid=technology.egit]).  
  
 
Note that the log found at this URL is marked "Tentative". This is automatically-generated content that is only considered official when it is actually submitted. To submit the log for review, you must log-in (using the link at the bottom of the page), correct any errors that you find (see below), and click "Submit".  
 
Note that the log found at this URL is marked "Tentative". This is automatically-generated content that is only considered official when it is actually submitted. To submit the log for review, you must log-in (using the link at the bottom of the page), correct any errors that you find (see below), and click "Submit".  
  
 
Note that you can adjust the set of projects that contribute data to the log by clicking the "Adjust" ("the set of projects to match the upcoming release") button. A log can be submitted for multiple projects; you might do this if your project provides a "roll-up" release and download that includes code from one or more subprojects.
 
Note that you can adjust the set of projects that contribute data to the log by clicking the "Adjust" ("the set of projects to match the upcoming release") button. A log can be submitted for multiple projects; you might do this if your project provides a "roll-up" release and download that includes code from one or more subprojects.
 
Note that, while you can submit a log for multiple projects, you can only edit the log for a single project. If you are doing a roll-up release, first edit the logs for each of the individual projects, and then submit the log for the combination.
 
  
 
When the log is submitted, it is reviewed by the IP Team. They may have questions, or ask for clarifications. You may be asked to resubmit the log after you have made some changes. When the process is complete, the IP Team will email you a PDF that contains the official log; you should make this available for your consumers and adopters to review.
 
When the log is submitted, it is reviewed by the IP Team. They may have questions, or ask for clarifications. You may be asked to resubmit the log after you have made some changes. When the process is complete, the IP Team will email you a PDF that contains the official log; you should make this available for your consumers and adopters to review.
 
==Correcting Errors In General==
 
In order to correct errors in the IP Log, you must:
 
# Be a committer on the project
 
# Be looking at an IP Log for a single project (because if there are multiple projects, we can't figure out where to record the corrections)
 
# Have the same email address for your Foundation email and your bugzilla email.
 
# Be logged in to the Eclipse website. The login mechanism uses the Friends of Eclipse login, so when you click "login" in the bottom menu, you will be taken to the Friends of Eclipse login. You do not need to be a Friend to use this login page: use your bugzilla login.
 
 
<center>[[Image:Iplog-correcting-errors-1.png]]</center>
 
 
When all of that is true, extra ''correction'' form fields will appear on the page along with a "Make These Corrections" submit button at the bottom. Like any html form, fill it out and press the button to save the corrections.
 
  
 
==Third-Party Code==
 
==Third-Party Code==
Line 60: Line 71:
  
 
==Unused Third-Party Code==
 
==Unused Third-Party Code==
If the project has CQs that approve the use of certain third-party code, but that code is not
+
If the project has CQs that approve the use of certain third-party code, but that code is not currently being used for this release, it is listed in the separate Unused Third-Party Code table. The CQs listed in this table are those that have one of the following keywords:
currently being used for this release, it is listed in the separate Unused Third-Party Code table.
+
The CQs listed in this table are those that have one of the following keywords:
+
 
* obsolete - this third-party code was used in previous releases, but it has been superseded or is obsolete and thus is no longer being used in this release and there is no plan to use it in a future release.
 
* obsolete - this third-party code was used in previous releases, but it has been superseded or is obsolete and thus is no longer being used in this release and there is no plan to use it in a future release.
 
* unused - the project requested approval of this third-party code in anticipation of using this code, but at this time the project is not using this code in this release. The project reserves the right to use the code in a future release.
 
* unused - the project requested approval of this third-party code in anticipation of using this code, but at this time the project is not using this code in this release. The project reserves the right to use the code in a future release.
Line 92: Line 101:
  
 
<u>Correcting Errors</u><br>
 
<u>Correcting Errors</u><br>
* '''Should Be Active'''. If a committer is listed as "Never Active" but should be listed in the active "Past and Present" table, then you should check the project meta data via [http://portal.eclipse.org/ the portal] - there are directories missing from the "source repository" item. If the project meta data is correct, you can use the "make active" checkbox to submit a correction. <center>[[Image:Iplog-correcting-errors-4.png]] <p><em>-or-</em></p> [[Image:Iplog-correcting-errors-2.png]]</center>
+
 
* '''Question Mark'''. If a "xxx???" person is listed as a committer, that means that the unix login "xxx" committed code to the source code repository, but there is no record of "xxx" as having been a committer on the project. This is a potential IP cleanliness problem, so contact the EMO to resolve it (do not just use the "exclude" checkbox correction without understanding why the data is incorrect).
+
If you need to make corrections to this section, then you may need to [[#Manually Generate the IP Log|manually generate the IP Log]].
* '''Extra Committer'''. If someone is listed as a "Never Active" committer, but you know that they were never even a committer at all, then there is an incorrect record in the Foundation database - contact the EMO. On the other hand, if someone is listed as a "Past and Present" committer, but you know that they were never a committer, then something larger is awry because there are source code check-ins by that person. Use the [http://dash.eclipse.org/dash/commits/web-app/summary.cgi commits explorer] to determine what check-ins they made and contact the EMO if there are any errors. In both cases, after resolving the issue with the EMO, use the "exclude" checkbox to correct the IP log. <center>[[Image:Iplog-correcting-errors-3.png]]</center>
+
 
* '''Missing Committer'''. If someone who is/was a committer with a unix account and is missing from both the "Past and Present" and the "Never Active" tables, then the first thing to check is the "source repository" project meta data item via [http://portal.eclipse.org/ the portal] - usually the problem is one or more unlisted source directories. <center>[[Image:Iplog-correcting-errors-4.png]]</center>
+
* '''Should Be Active'''. If a committer is listed as "Never Active" but should be listed in the active "Past and Present" table, then you should confirm that the project metadata correctly lists all project source code repositories. The easiest way to a missing committer is to have them make a commit (in any branch).  
 +
* '''Question Mark'''. If a "xxx???" person is listed as a committer, that means that the unix login "xxx" committed code to the source code repository, but there is no record of "xxx" as having been a committer on the project. This is a potential IP cleanliness problem, so contact the [mailto:emo@eclipse.org EMO] to resolve it.
 +
* '''Extra Committer'''. If someone is listed as a "Never Active" committer, but you know that they were never even a committer at all, then there is an incorrect record in the Foundation database - contact the [mailto:emo@eclipse.org EMO]. On the other hand, if someone is listed as a "Past and Present" committer, but you know that they were never a committer, then something larger is awry because there are source code check-ins by that person. Use the [http://dash.eclipse.org/dash/commits/web-app/summary.cgi commits explorer] to determine what check-ins they made and contact the [mailto:emo@eclipse.org EMO] if there are any errors.
 +
* '''Missing Committer'''. If someone who is/was a committer with a unix account and is missing from both the "Past and Present" and the "Never Active" tables, then confirm that the project metadata correctly lists all project source code repositories and contact the [mailto:emo@eclipse.org EMO]
  
 
==Contributors==
 
==Contributors==
The Contributors section of the IP Log uses one information source: bugzilla (driven by the project meta data <code>bugzilla</code> item(s))
+
The Contributors section of the IP Log uses two information sources: bugzilla (driven by the project meta data <code>bugzilla</code> item(s)) and Git repositories (specified by the project metadata <code>sourcerepositories</code> items.
 +
 
 +
For '''Bugzilla''', the automatic IP Log considers:
 
# all people who were not committers when they attached:
 
# all people who were not committers when they attached:
 
## an attachment to a bug
 
## an attachment to a bug
Line 110: Line 124:
  
 
In plain English, this means there are two different ways to mark a contribution in bugzilla. If the contribution is an attachment to a bug such as a patch, the attachment itself should be marked with the iplog+ flag.  If the contribution is in the form of a comment in the bug, then the iplog+ flag should be set on the bug itself. Since this will mark all non-committers who commented on the bug as contributors, you typically then need to use the exceptions mechanism to exclude comments that are not contributions. Typically these two mechanisms are mutually exclusive: if the contribution is an attachment, you do not also need to set iplog+ flag on the bug itself (unless the bug also has other contributions in the form of comments).
 
In plain English, this means there are two different ways to mark a contribution in bugzilla. If the contribution is an attachment to a bug such as a patch, the attachment itself should be marked with the iplog+ flag.  If the contribution is in the form of a comment in the bug, then the iplog+ flag should be set on the bug itself. Since this will mark all non-committers who commented on the bug as contributors, you typically then need to use the exceptions mechanism to exclude comments that are not contributions. Typically these two mechanisms are mutually exclusive: if the contribution is an attachment, you do not also need to set iplog+ flag on the bug itself (unless the bug also has other contributions in the form of comments).
 +
 +
For contributions made through '''Git''', the automatic IP Log grabs all commit records with a author who is not a project committer. For more information, please see [[Development Resources/Handling Git Contributions|Handling Git Contributions]].
  
 
<u>Correcting Errors</u><br>
 
<u>Correcting Errors</u><br>
* '''No Name.''' If the contributor record for person X shows an email address but no name, then person X has not entered a name in bugzilla. Have X go to bugzilla Preferences and enter their correct name. <center>[[Image:Iplog-correcting-errors-7.png]]</center>
+
* '''No Name.''' If the contributor record for person X shows an email address but no name, then person X has not entered a name in Bugzilla. Have X go to bugzilla Preferences and enter their correct name. <center>[[Image:Iplog-correcting-errors-7.png]]</center>
* '''Error Messages.''' If the tentative IP log shows errors (red error messages) in product name or component name, then the team should check their project meta data via [http://portal.eclipse.org the portal] - the product name and (optional) component names must be an exact match with bugzilla, capitalization and all.
+
* '''Error Messages.''' If the tentative IP log shows errors (red error messages) in product name or component name, then the team should check the Bugzilla entries in the project metadata - the product name and (optional) component names must be an exact match with Bugzilla, capitalization and all.
* '''Extra Contributor.''' If a person (person X) is listed as a Contributor but the team knows that their patch was never used/is not currently in use, there are three solutions: (1; preferred) remove the "iplog+" flag from the bugzilla attachment, or (2) use "exclude" correction to modify the IP log (if contributor status is based on a comment instead of an attachment). <center>[[Image:Iplog-correcting-errors-6.png]]<p><em>-or-</em></p>[[Image:Iplog-correcting-errors-5.png]]</center>
+
* '''Extra Contributor.''' If a person (person X) is listed as a Contributor but the team knows that their patch was never used/is not currently in use, there are three solutions: (1; preferred) remove the "iplog+" flag from the bugzilla attachment.
 
* '''Missing Contributor.''' If person X should be listed as a Contributor but is not, then be sure that person X has attached a patch to a fixed bug and that patch is flagged with "iplog+". All external code contributions should come through bugzilla patches, so if a code contribution first comes through email, the contribution must also add a patch. Note that person X must add the patch because if person Y adds the patch for person X, then person Y will be listed as the contributor. <center>[[Image:Iplog-correcting-errors-6.png]]</center>
 
* '''Missing Contributor.''' If person X should be listed as a Contributor but is not, then be sure that person X has attached a patch to a fixed bug and that patch is flagged with "iplog+". All external code contributions should come through bugzilla patches, so if a code contribution first comes through email, the contribution must also add a patch. Note that person X must add the patch because if person Y adds the patch for person X, then person Y will be listed as the contributor. <center>[[Image:Iplog-correcting-errors-6.png]]</center>
 
* '''Committer is Listed.''' If committer X is listed as a Contributor, check the following:
 
* '''Committer is Listed.''' If committer X is listed as a Contributor, check the following:
Line 122: Line 138:
 
==Keeping the IP Log Current==
 
==Keeping the IP Log Current==
 
Keeping the automatic IP log current is fairly simple:
 
Keeping the automatic IP log current is fairly simple:
# Make sure that all third-party code and dependencies are filed as CQs.
+
# Make sure that all third-party code and dependencies are filed as CQs. You can use this [http://www.eclipse.org/projects/tools/downloads.php script] to view all content in your download directory organized by CQ. Anything showing up red on this scan may need a CQ.
 
# If a non-committer patch is accepted, mark it with the iplog+ flag.
 
# If a non-committer patch is accepted, mark it with the iplog+ flag.
 
# If a non-committer bug without a patch is accepted, mark the whole bug with the iplog+ flag and then go to the automatic IP log to exclude the non-contribution comments.
 
# If a non-committer bug without a patch is accepted, mark the whole bug with the iplog+ flag and then go to the automatic IP log to exclude the non-contribution comments.
Line 141: Line 157:
 
==Problems with the IP Log?==
 
==Problems with the IP Log?==
 
If you're having trouble updating the IP log or if the IP log is not showing enough information, please file a bug (use [https://bugs.eclipse.org/bugs/enter_bug.cgi?assigned_to=portal-inbox%40eclipse.org&blocked=&bug_file_loc=http%3A%2F%2F&bug_severity=normal&bug_status=NEW&comment=&component=Portal&contenttypeentry=&contenttypemethod=autodetect&contenttypeselection=text%2Fplain&data=&dependson=&description=&flag_type-1=X&flag_type-2=X&flag_type-4=X&flag_type-6=X&flag_type-7=X&flag_type-8=X&form_name=enter_bug&keywords=&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=All&priority=P3&product=Community&qa_contact=&rep_platform=All&short_desc=&version=unspecified this link]) and we'll work hard to make things right.
 
If you're having trouble updating the IP log or if the IP log is not showing enough information, please file a bug (use [https://bugs.eclipse.org/bugs/enter_bug.cgi?assigned_to=portal-inbox%40eclipse.org&blocked=&bug_file_loc=http%3A%2F%2F&bug_severity=normal&bug_status=NEW&comment=&component=Portal&contenttypeentry=&contenttypemethod=autodetect&contenttypeselection=text%2Fplain&data=&dependson=&description=&flag_type-1=X&flag_type-2=X&flag_type-4=X&flag_type-6=X&flag_type-7=X&flag_type-8=X&form_name=enter_bug&keywords=&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=All&priority=P3&product=Community&qa_contact=&rep_platform=All&short_desc=&version=unspecified this link]) and we'll work hard to make things right.
 +
 +
=Manually Generate the IP Log=
 +
 +
==From HTML==
 +
 +
You can generate a pure XML-version of the log by adding the <code>generatefinal=true</code> parameter to the URL, i.e. <nowiki>http://www.eclipse.org/projects/ip_log.php?id=<project&nbsp;id>&generatefinal=true</nowiki>.
 +
 +
Use either wget, or simply "Save as..." from your browser to save a local copy and make whatever changes are necessary.
 +
 +
Submit the HTML version only via email to [mailto:emo@eclipse.org,emo-ipteam@eclipse.org?subject=Request%20IP%20Log%20Review%20for%20projectid the EMO and EMO IP team] with a short note requesting the review and describing (in broad terms) how you've modified the generated log. Please include the id of your project in the subject line. If the IP Log is being submitted for a release review, please indicate the version of the release in the message text.
 +
 +
==From XML==
 +
 +
An XML file (schema: [http://www.eclipse.org/projects/xml/iplog.xsd iplog.xsd]) containing the IP Log data can be generated via a RESTful web services call to <nowiki>http://eclipse.org/projects/xml/iplog.xml.php?id=<project&nbsp;id></nowiki> (where "project id" is--oddly enough--the id of your project; e.g. technology.egit). You an use your XML editor of choice to make the necessary modifications.
 +
 +
Use the [http://www.eclipse.org/projects/xml/iplog.xsl iplog.xsl] file to convert the modified IP Log to HTML.
 +
 +
For example (from Linux using xsltproc):
 +
 +
<nowiki>xsltproc http://eclipse.org/projects/xml/iplog.xsl http://eclipse.org/projects/xml/iplog.xml.php?id=technology.egit > technology.egit-3.2-iplog.html</nowiki>
 +
 +
Submit the HTML version only via email to [mailto:emo@eclipse.org,emo-ipteam@eclipse.org?subject=Request%20IP%20Log%20Review%20for%20projectid the EMO and EMO IP team] with a short note requesting the review and describing (in broad terms) how you've modified the generated log. Please include the id of your project in the subject line. If the IP Log is being submitted for a release review, please indicate the version of the release in the message text.
  
 
''This page is moderated by the EMO''
 
''This page is moderated by the EMO''
 
[[Category:Development_Resources]]
 
[[Category:Development_Resources]]

Revision as of 05:03, 1 April 2014

The Foundation automatically produces IP Logs for each project (bug 220977). This page describes how the automatic IP Log is generated (by the system) and maintained (by the project team) and processed (by Eclipse Legal).

What is an IP Log?

An IP Log is a record of the intellectual property (IP) contributions to a project. This includes such as a list of all committers, past and present, that have worked on the code and (especially) those who have made contributions to the current code base. The expected contents of a log is described on the Project IP Log page.

What is the Automated IP Log Tool?

The Automated IP Log Tool automatically generates an IP Log using information that is available to the Eclipse Foundation. The list of committers, for example is generated using information provided by the Dash project which itself pulls information out of source code repositories.

IPLogGenerator.png

The IP Log generator pulls information from multiple location to assemble the log:

  • Third-party libraries used by the project come from IPZilla
  • The Dash process scans the project source code repositories to assess committer activity
  • Dash also scans Git repositories for contributions
    • If you follow the guidelines for handling Git contributions, contributions received via Git in any branch will automatically appear in the log
    • Dash determines repository ownership from the project metadata; this same information is used to populate the list of repositories in the log
  • Contributions received as patches in Bugzilla that are marked iplog+ will automatically appear in the log
    • Contributions flagged in Bugzilla will only appear in the IP Log if the bug is marked FIXED or CLOSED.
  • License information is obtained from the Foundation database
Warning2.png
Dash gathers its information via a long-running process that is initiated weekly (on Saturday night). If you make changes to the data that Dash uses to run, those changes will not be reflected immediately. If necessary, the scripts can be initiated manually by the EMO if required (ask nicely).


To fully leverage the value of the Automated IP Log Tool, you need to:

Warning2.png
Contributions should be recorded in one of Git or Bugzilla. The IP Log generator is not smart enough to detect duplicate entries.


Your project's metadata is used to determine the identities of the source code repositories that Dash needs to scan to find out committer information. Specifically, you need to specify, in the "Source Repositories" section, a list of paths to source code repository locations. With this information specified, not only will the Automated IP Log tool be able to build the "Committers" section of the IP Log for you, but your project's Project Summary page will also be updated to accurately reflect your project's commit activity.

The Automated IP Log tool populates the Developers section with information gathered from Git and Bugzilla. This section lists contributions from non-committers (this is time-sensitive, so contributions made by current committers before they became committers will also be included).

Git commits contributed by non-committers are identified by the author identity on the commit record; the "Author" and "Author Name" fields must be set to the identity of the actual author of the commit.

Only non-committer contributors are recorded in the generated log. To aid the tool in generating correct results, you need to mark Bugzilla attachments with the iplog+ flag. In setting this flag, the Automated IP Log tool assumes that the person who attached the bug is the contributor. To comply with the website terms of use, the person who attaches the contribution really needs to be the person who has permission to make it available. You should ensure that this is the case before including the code in your project's repository and flagging the entry.

Alternatively, you can flag an entire Bugzilla entry with +iplog. Doing so, however, indicates to the Automated IP Log tool that every single comment made by a non-committer in the bug report represents a potential contribution. For your own sanity, it's a good practice to ask contributors to provide and attach patches that can be individually marked. Marking an entire bug represents an ongoing maintenance issue as new comments added to the bug from non-committers will show up in the generated log.

Note that contributions flagged in Bugzilla will only appear in the IP Log if the bug is marked FIXED or CLOSED.

The Bugzilla Contribution Review Tool can help you identify potential contributions.

The Third-Party Software section of the log is populated from our IPZilla Bugzilla instance. The IP Team will mark your contributions in such a way that they will appear in the log. If third party software is not appearing properly, contact the EMO IP Team to make corrections.

Using the Automated IP Log

Warning2.png
The Automated IP Log's editing functionality is broken, considered retired (though not completely removed), and due for replacement/integration with the Project Management Infrastructure. It has been our experience that the edit functionality is rarely (if ever) actually required; problems that require editing can almost always be addressed by correcting the underlying data.


To start with the Automated IP Log, you need to first navigate to the page. The URL is of the form: http://www.eclipse.org/projects/ip_log.php?projectid=[projectId] (e.g. [1]).

Note that the log found at this URL is marked "Tentative". This is automatically-generated content that is only considered official when it is actually submitted. To submit the log for review, you must log-in (using the link at the bottom of the page), correct any errors that you find (see below), and click "Submit".

Note that you can adjust the set of projects that contribute data to the log by clicking the "Adjust" ("the set of projects to match the upcoming release") button. A log can be submitted for multiple projects; you might do this if your project provides a "roll-up" release and download that includes code from one or more subprojects.

When the log is submitted, it is reviewed by the IP Team. They may have questions, or ask for clarifications. You may be asked to resubmit the log after you have made some changes. When the process is complete, the IP Team will email you a PDF that contains the official log; you should make this available for your consumers and adopters to review.

Third-Party Code

The Third-Party Code section of the IP Log shows the approved Contribution Questionnaires from ipzilla. The CQs listed in this table are those that:

  • state=approved, or
  • RESOLVED/FIXED or CLOSED/FIXED (regardless of state), and
  • contain the keyword "nonepl"

Correcting Errors

  • Missing CQ. If a CQ is missing from this list, (a) check its status, resolution, and state: make sure that they match one of the above cases, and (b) check to ensure "nonepl" keyword is included in order to identify the code as third party. If not, contact Eclipse Legal.
  • Extra CQ. If a CQ is listed but does not belong to this project, then contact Eclipse Legal to have the ipzilla database corrected.
  • Missing or Incorrect License. If a CQ is missing the license information, contact Eclipse Legal to have the correct license information added to the database.

Unused Third-Party Code

If the project has CQs that approve the use of certain third-party code, but that code is not currently being used for this release, it is listed in the separate Unused Third-Party Code table. The CQs listed in this table are those that have one of the following keywords:

  • obsolete - this third-party code was used in previous releases, but it has been superseded or is obsolete and thus is no longer being used in this release and there is no plan to use it in a future release.
  • unused - the project requested approval of this third-party code in anticipation of using this code, but at this time the project is not using this code in this release. The project reserves the right to use the code in a future release.
  • withdrawn - the project requested approval of this third-code code but subsequent to receiving approval and prior to using the code, decided not to use said code. This code has not been used in any release, is not being used in this release, and there is no plan to use it in a future release.

Correcting Errors

  • CQ Is Being Used. If a CQ is listed as unused but is actually being used, contact Eclipse Legal to have the keywords on CQ removed/changed.
  • CQ Should Be Unused. If a CQ should be in the unused list instead of one of the other lists, contact Eclipse Legal to have the appropriate keyword added.

Works With and Pre-Req Dependencies

If the project has Works With, Pre-Req, or Exempt Pre-Req Dependencies, these will be listed in a separate table. These are also Contribution Questionnaires from ipzilla. These CQs are:

  • state=exempt_prereq
  • state=prereq
  • state=workswith

Correcting Errors

  • Missing Pre-Req. If an pre-req is missing from this list, first check that to see if there is a CQ for it. If there is no CQ, submit one (using the portal). If there is a CQ, check that its state is "workswith", "prereq", or "exempt_prereq" - if not, contact Eclipse Legal.

Pending Contribution Questionnaires

If the project has any pending and not yet approved or rejected Contribution Questionnaires, these are listed in a separate table. These represent potential IP issues and thus must be considered carefully during a Release Review.

Correcting Errors

  • Unneeded CQ. If the code represented by a pending CQ is no longer needed by the project, use ipzilla to withdraw the CQ.

Committers

The Committers section of the IP Log uses two information sources to generate the two lists:

  1. The list of all committers who have committed code to any of the source repositories from the Commits Explorer. This is the basis of the "Past and Present" list. The commits explorer list is driven by the project meta data source repository item(s).
  2. The list of all past and present committers from the Foundation's internal database of committers and projects. Committers here but not in in the "Past and Present" list are put on the "Never Active" list.

Correcting Errors

If you need to make corrections to this section, then you may need to manually generate the IP Log.

  • Should Be Active. If a committer is listed as "Never Active" but should be listed in the active "Past and Present" table, then you should confirm that the project metadata correctly lists all project source code repositories. The easiest way to a missing committer is to have them make a commit (in any branch).
  • Question Mark. If a "xxx???" person is listed as a committer, that means that the unix login "xxx" committed code to the source code repository, but there is no record of "xxx" as having been a committer on the project. This is a potential IP cleanliness problem, so contact the EMO to resolve it.
  • Extra Committer. If someone is listed as a "Never Active" committer, but you know that they were never even a committer at all, then there is an incorrect record in the Foundation database - contact the EMO. On the other hand, if someone is listed as a "Past and Present" committer, but you know that they were never a committer, then something larger is awry because there are source code check-ins by that person. Use the commits explorer to determine what check-ins they made and contact the EMO if there are any errors.
  • Missing Committer. If someone who is/was a committer with a unix account and is missing from both the "Past and Present" and the "Never Active" tables, then confirm that the project metadata correctly lists all project source code repositories and contact the EMO

Contributors

The Contributors section of the IP Log uses two information sources: bugzilla (driven by the project meta data bugzilla item(s)) and Git repositories (specified by the project metadata sourcerepositories items.

For Bugzilla, the automatic IP Log considers:

  1. all people who were not committers when they attached:
    1. an attachment to a bug
    2. and the attachment is flagged with "iplog+"
    3. and is attached to a RESOLVED/FIXED, VERIFIED/FIXED, or CLOSED/FIXED bug
    For example, see bug 69671; the contribution is in a flagged attachment so it shows up in the CDT IP Log.
  2. all people who were not committers when they commented on a bug:
    1. that is flagged with "iplog+"
    2. and RESOLVED/FIXED, VERIFIED/FIXED, or CLOSED/FIXED
    For example, see bug 190154 in which the contribution is in comments #0 and #11. Then see the BIRT IP Log. Note that comments #1-10 and #12 were excluded by the project lead using the exceptions mechanism.

In plain English, this means there are two different ways to mark a contribution in bugzilla. If the contribution is an attachment to a bug such as a patch, the attachment itself should be marked with the iplog+ flag. If the contribution is in the form of a comment in the bug, then the iplog+ flag should be set on the bug itself. Since this will mark all non-committers who commented on the bug as contributors, you typically then need to use the exceptions mechanism to exclude comments that are not contributions. Typically these two mechanisms are mutually exclusive: if the contribution is an attachment, you do not also need to set iplog+ flag on the bug itself (unless the bug also has other contributions in the form of comments).

For contributions made through Git, the automatic IP Log grabs all commit records with a author who is not a project committer. For more information, please see Handling Git Contributions.

Correcting Errors

  • No Name. If the contributor record for person X shows an email address but no name, then person X has not entered a name in Bugzilla. Have X go to bugzilla Preferences and enter their correct name.
    Iplog-correcting-errors-7.png
  • Error Messages. If the tentative IP log shows errors (red error messages) in product name or component name, then the team should check the Bugzilla entries in the project metadata - the product name and (optional) component names must be an exact match with Bugzilla, capitalization and all.
  • Extra Contributor. If a person (person X) is listed as a Contributor but the team knows that their patch was never used/is not currently in use, there are three solutions: (1; preferred) remove the "iplog+" flag from the bugzilla attachment.
  • Missing Contributor. If person X should be listed as a Contributor but is not, then be sure that person X has attached a patch to a fixed bug and that patch is flagged with "iplog+". All external code contributions should come through bugzilla patches, so if a code contribution first comes through email, the contribution must also add a patch. Note that person X must add the patch because if person Y adds the patch for person X, then person Y will be listed as the contributor.
    Iplog-correcting-errors-6.png
  • Committer is Listed. If committer X is listed as a Contributor, check the following:
    1. First, a committer can be listed as a contributor if he or she was a contributor, contributed some code, and then later became a committer. He/she will be listed as a contributor for the patches he/she submitted before becoming a committer.
    2. Check that person X's email address in bugzilla matches any one of their email addresses on file with the Foundation. Have person X use the the portal to check to verify their email addresses. If their bugzilla address does not match the Foundation committer records, we have no way to associate their bugzilla records with their committer status.

Keeping the IP Log Current

Keeping the automatic IP log current is fairly simple:

  1. Make sure that all third-party code and dependencies are filed as CQs. You can use this script to view all content in your download directory organized by CQ. Anything showing up red on this scan may need a CQ.
  2. If a non-committer patch is accepted, mark it with the iplog+ flag.
  3. If a non-committer bug without a patch is accepted, mark the whole bug with the iplog+ flag and then go to the automatic IP log to exclude the non-contribution comments.
  4. Use the portal to deactivate inactive committers

Choosing the Set of Projects

The simplest case is an IP Log for a single project. However, many project teams work together to create a unified release across a set of projects. In this case, the IP Log must also be an aggregated/unified IP Log across the same set of projects. At the bottom left of the IP Log page is a "adjust the set of projects" link. That link leads to the selector page. Use the checkboxes to select a set of projects and generate a unified IP Log for all of those projects.

Iplog-correcting-errors-8.png

Submitting the IP Log to Eclipse Legal

Single or unified-multi-project, either way the IP Log can be submitted to Eclipse Legal for review using the link at bottom center of the page. Click that, fill out the reason, and voila, the a frozen copy of the log is sent to Legal.

Iplog-correcting-errors-9.png

You must send your IP log to Eclipse Legal at least three weeks in advance of your scheduled Release Review to guarantee your review date. Sending your IP log in late will jeopardize your review date due to Eclipse Legal's workload.

Problems with the IP Log?

If you're having trouble updating the IP log or if the IP log is not showing enough information, please file a bug (use this link) and we'll work hard to make things right.

Manually Generate the IP Log

From HTML

You can generate a pure XML-version of the log by adding the generatefinal=true parameter to the URL, i.e. http://www.eclipse.org/projects/ip_log.php?id=<project id>&generatefinal=true.

Use either wget, or simply "Save as..." from your browser to save a local copy and make whatever changes are necessary.

Submit the HTML version only via email to the EMO and EMO IP team with a short note requesting the review and describing (in broad terms) how you've modified the generated log. Please include the id of your project in the subject line. If the IP Log is being submitted for a release review, please indicate the version of the release in the message text.

From XML

An XML file (schema: iplog.xsd) containing the IP Log data can be generated via a RESTful web services call to http://eclipse.org/projects/xml/iplog.xml.php?id=<project id> (where "project id" is--oddly enough--the id of your project; e.g. technology.egit). You an use your XML editor of choice to make the necessary modifications.

Use the iplog.xsl file to convert the modified IP Log to HTML.

For example (from Linux using xsltproc):

xsltproc http://eclipse.org/projects/xml/iplog.xsl http://eclipse.org/projects/xml/iplog.xml.php?id=technology.egit > technology.egit-3.2-iplog.html

Submit the HTML version only via email to the EMO and EMO IP team with a short note requesting the review and describing (in broad terms) how you've modified the generated log. Please include the id of your project in the subject line. If the IP Log is being submitted for a release review, please indicate the version of the release in the message text.

This page is moderated by the EMO

Back to the top