From: armando-migliaccio Date: Tue, 29 Sep 2015 05:55:44 +0000 (-0700) Subject: Changes in Neutron defect management X-Git-Url: https://review.fuel-infra.org/gitweb?a=commitdiff_plain;h=06298824f09d0e276161926561674aa9018a2aa8;p=openstack-build%2Fneutron-build.git Changes in Neutron defect management This patch proposes some changes to the way we manage bugs in Neutron, and how defect management is to be shared across the Neutron team. Change-Id: I51bf74cef11dc17b88c18a2007306b913da9f981 --- diff --git a/doc/source/policies/bugs.rst b/doc/source/policies/bugs.rst index 5c1aaf238..b2a5d6ce7 100644 --- a/doc/source/policies/bugs.rst +++ b/doc/source/policies/bugs.rst @@ -1,45 +1,602 @@ Neutron Bugs ============ -Neutron maintains all of it's bugs in `Launchpad `_. All of -the current open Neutron bugs can be found in that link. +Neutron (client, core, FwaaS, LBaaS, VPNaaS) maintains all of its bugs in the following +Launchpad projects: -Neutron Bug Czar ----------------- -Neutron maintains the notion of a "bug czar." The bug czar plays an important role in the Neutron -community. As a large project, Neutron is routinely fielding many bug reports. The bug czar is -responsible for acting as a "first contact" for these bug reports and performing initial -triaging. The bug czar is expected to communicate with the various Neutron teams when a bug has -been triaged. In addition, the bug czar should be reporting "High" and "Critical" priority bugs -to both the PTL and the core reviewer team during each weekly Neutron meeting. +* `Launchpad Neutron `_ +* `Launchpad python-neutronclient `_ + + +Neutron Bug Deputy +------------------ + +Neutron maintains the notion of a "bug deputy". The bug deputy plays an +important role in the Neutron community. As a large project, Neutron is +routinely fielding many bug reports. The bug deputy is responsible for +acting as a "first contact" for these bug reports and performing initial +screening/triaging. The bug deputy is expected to communicate with the +various Neutron teams when a bug has been triaged. In addition, the bug +deputy should be reporting "High" and "Critical" priority bugs. + +To avoid burnout, and to give a chance to everyone to gain experience in +defect management, the Neutron bug deputy is a rotating role: every member +of the Neutron core team is invited to fill in the role, and the rotation +will be set on a period (typically one or two weeks) determined by the team +during the weekly Neutron IRC meeting and/or according to holidays. During +the Neutron IRC meeting we will expect a volunteer to step up for the period. +Non-core volunteers are encouraged in taking up the role. + +This contributor is going to be the bug deputy for the period, and he/she +will be asked to report to the team during the subsequent IRC meeting. The +PTL will also work with the team to assess that everyone gets his/her fair +share at fulfilling this duty. It is reasonable to expect some imbalance +from time to time, and the team will work together to resolve it to ensure +that everyone is 100% effective and well rounded in their role as +_custodian_ of Neutron quality. Should the duty load be too much in busy +times of the release, the PTL and the team will work together to assess +whether more than one deputy is necessary in a given period. + +The presence of a bug deputy does not mean the rest of the team is simply off +the hook for the period, in fact the bug deputy will have to actively work +with the Lieutenants/Drivers, and these should help in getting the bug report +moving down the resolution pipeline. + +During the period a member acts as bug deputy, he/she is expected to watch +bugs filed against the Neutron projects (as listed above) and do a first +screening to determine potential severity, tagging, logstash queries, other +affected projects, affected releases, etc. + +From time to time bugs will be filed and auto-assigned by members of the +core team to get them to a swift resolution. Obviously, the deputy is exempt +from screening these. + +Finally, the PTL will work with the deputy to produce a brief summary of the +issues of the week to be shared with the larger team during the weekly IRC +meeting and tracked in the meeting notes. -The current Neutron bug czar is Kyle Mestery (IRC nick mestery). Plugin and Driver Repositories ------------------------------ Many plugins and drivers have backend code that exists in another repository. -These repositories have their own launchpad projects for bugs. The teams +These repositories may have their own Launchpad projects for bugs. The teams working on the code in these repos assume full responsibility for bug handling -in those projects. +in those projects. For this reason, bugs whose solution would exist solely in +the plugin/driver repo should not have Neutron in the affected projects section. +However, you should add Neutron (Or any other project) to that list only if you +expect that a patch is needed to that repo in order to solve the bug. + +It also worth adding that that some of these projects are part of the so +called Neutron `stadium `_. +Because of that, their release is managed centrally by the Neutron +release team; requests for releases need to funnelled and screened +properly before they can happen. To this aim, the process should be like +the following: + +* Create a bug report to your Launchpad project: provide details as to what + you would like to release; +* Add Neutron to the list of affected projects. +* Add 'release-subproject' tag to the list of tags for the bug report. +* The Neutron release management team will watch these bugs, and work with + you to have the request fulfilled. + + +.. _guidelines: + +Bug Screening Best Practices +---------------------------- + +When screening bug reports, the first step for the bug deputy is to assess +how well written the bug report is, and whether there is enough information +for anyone else besides the bug submitter to reproduce the bug and come up +with a fix. There is plenty of information on the `OpenStack wiki `_ +on how to write a good bug `report `_ +and to learn how to tell a good bug report from a bad one. Should the bug +report not adhere to these best practices, the bug deputy's first step +would be to redirect the submitter to this section, invite him/her to supply +the missing information, and mark the bug report as 'Incomplete'. For future +submissions, the reporter can then use the template provided below to ensure +speedy triaging. Done often enough, this practice should (ideally) ensure that +in the long run, only 'good' bug reports are going to be filed. + +Bug Report Template +~~~~~~~~~~~~~~~~~~~ + +The more information you provide, the higher the chance of speedy triaging and +resolution: identifing the problem is half the solution. To this aim, when +writing a bug report, please consider supplying the following details and +following these suggestions: + +* Summary (Bug title): keep it small, possibly one line. If you cannot describe + the issue in less than 100 characters, you are probably submitting more than + one bug at once. +* Further information (Bug description): conversely from other bug trackers, + Launchpad does not provide a structured way of submitting bug-related + information, but everything goes in this section. Therefore, you are invited + to break down the description in the following fields: + + * High level description: provide a brief sentence (a couple of lines) of + what are you trying to accomplish, or would like to accomplish differently; + the 'why' is important, but can be omitted if obvious (not to you of course). + * Pre-conditions: what is the initial state of your system? Please consider + enumerating resources available in the system, if useful in diagnosing + the problem. Who are you? A regular tenant or a super-user? Are you + describing service-to-service interaction? + * Step-by-step reproduction steps: these can be actual neutron client + commands or raw API requests; Grab the output if you think it is useful. + Please, consider using `paste.o.o `_ for long + outputs as Launchpad poorly format the description field, making the + reading experience somewhat painful. + * Expected output: what did you hope to see? How would you have expected the + system to behave? A specific error/success code? The output in a specific + format? Or more than a user was supposed to see, or less? + * Actual output: did the system silently fail (in this case log traces are + useful)? Did you get a different response from what you expected? + * Version: + + * OpenStack version (Specific stable branch, or git hash if from trunk); + * Linux distro, kernel. For a distro, it's also worth knowing specific + versions of client and server, not just major release; + * Relevant underlying processes such as openvswitch, iproute etc; + * DevStack or other _deployment_ mechanism? + + * Environment: what services are you running (core services like DB and + AMQP broker, as well as Nova/hypervisor if it matters), and which type + of deployment (clustered servers); if you are running DevStack, is it a + single node? Is it multi-node? Are you reporting an issue in your own + environment or something you encountered in the OpenStack CI + Infrastructure, aka the Gate? + * Perceived severity: what would you consider the `importance `_ + to be? + +* Tags (Affected component): try to use the existing tags by relying on + auto-completion. Please, refrain from creating new ones, if you need + new "official" tags_, please reach out to the PTL. If you would like + a fix to be backported, please add a backport-potential tag. + This does not mean you are gonna get the backport, as the stable team needs + to follow the `stable branch policy `_ + for merging fixes to stable branches. +* Attachments: consider attaching logs, truncated log snippets are seldomly + useful. Be proactive, and consider attaching redacted configuration files + if you can, as that will speed up the resolution process greatly. + Bug Triage Process ------------------- +~~~~~~~~~~~~~~~~~~ The process of bug triaging consists of the following steps: -1. Check if a bug was filed for a correct component (project). If not, either change the project - or mark it as "Invalid". -2. Add appropriate tags. Even if the bug is not valid or is a duplicate of another one, it still - may help bug submitters and corresponding sub-teams. -3. Check if a similar bug was filed before. If so, mark it as a duplicate of the previous bug. -4. Check if the bug description is consistent, e.g. it has enough information for developers to - reproduce it. If it's not consistent, ask submitter to provide more info and mark a bug as - "Incomplete". -5. Depending on ease of reproduction (or if the issue can be spotted in the code), mark it as - "Confirmed". -6. Assign the importance. Bugs that obviously break core and widely used functionality should get - assigned as "High" or "Critical" importance. The same applies to bugs that were filed for gate - failures. -7. (Optional). Add comments explaining the issue and possible strategy of fixing/working around - the bug. +* Check if a bug was filed for a correct component (project). If not, either + change the project or mark it as "Invalid". +* Check if a similar bug was filed before. Rely on your memory if Launchpad + is not clever enough to spot a duplicate upon submission. If so, mark it + as a duplicate of the previous bug. +* Check if the bug meets the requirements of a good bug report, by checking + that the guidelines_ are being followed. Omitted information is still + acceptable if the issue is clear nonethless; use your good judgement and + your experience. Consult another core member/PTL if in doubt. If the bug + report needs some love, mark the bug as 'Incomplete', point the submitter + to this document and hope he/she turns around quickly with the missing + information. + +If the bug report is sound, move next: + +* Revise tags as recommended by the submitter. Ensure they are 'official' + tags. +* Depending on ease of reproduction (or if the issue can be spotted in the + code), mark it as 'Confirmed'. If you are unable to assess/triage the + issue because you do not have access to a repro environment, consider + reaching out the Lieutenant, go-to person for the affected component; + he/she may be able to help: assign the bug to him/her for further + screening. If the bug already has an assignee, check that a patch is + in progress. Sometimes more than one patch is required to address an + issue, make sure that there is at least one patch that 'Closes' the bug + or document/question what it takes to mark the bug as fixed. +* If the bug is the result of a misuse of the system, mark the bug either + as 'Won't fix', or 'Opinion' if you are still on the fence and need + other people's input. +* Assign the importance after reviewing the proposed severity. Bugs that + obviously break core and widely used functionality should get assigned as + "High" or "Critical" importance. The same applies to bugs that were filed + for gate failures. +* Choose a milestone, if you can. Targeted bugs are especially important + close to the end of the release. +* (Optional). Add comments explaining the issue and possible strategy of + fixing/working around the bug. Also, as good as some are at adding all + thoughts to bugs, it is still helpful to share the in-progress items + that might not be captured in a bug description or during our weekly + meeting. In order to provide some guidance and reduce ramp up time as + we rotate, tagging bugs with 'needs-attention' can be useful to quickly + identify what reports need further screening/eyes on. + +You are done! Iterate. + + +Bug Expiration Policy and Bug Squashing +--------------------------------------- + +More can be found at this `Launchpad page `_. +In a nutshell, in order to make a bug report expire automatically, it needs to be +unassigned, untargeted, and marked as Incomplete. + +The OpenStack community has had `Bug Days `_ +but they have not been wildly successful. In order to keep the list of open bugs set +to a manageable number (more like <100+, rather than closer to 1000+), at the end of +each release (in feature freeze and/or during less busy times), the PTL with the +help of team will go through the list of open (namely new, opinion, in progress, +confirmed, triaged) bugs, and do a major sweep to have the Launchpad Janitor pick +them up. This gives 60 days grace period to reporters/assignees to come back and +revive the bug. Assuming that at regime, bugs are properly reported, acknowledged +and fix-proposed, losing unaddressed issues is not going to be a major issue, +but brief stats will be collected to assess how the team is doing over time. + + +.. _tags: + +Tagging Bugs +------------ + +Launchpad's Bug Tracker allows you to create ad-hoc groups of bugs with tagging. + +In the Neutron team, we have a list of agreed tags that we may apply to bugs +reported against various aspects of Neutron itself. The list of approved tags +used to be available on the `wiki `_, +however the section has been moved here, to improve collaborative editing, and +keep the information more current. By using a standard set of tags, each +explained on this page, we can avoid confusion. A bug report can have more than +one tag at any given time. + +Proposing New Tags +~~~~~~~~~~~~~~~~~~ + +New tags, or changes in the meaning of existing tags (or deletion), are to be +proposed via patch to this section. After discussion, and approval, a member of +the bug team will create/delete the tag in Launchpad. Each tag covers an area +with an identified go-to contact or `Lieutenant `_, +who can provide further insight. Bug queries are provided below for convenience, +more will be added over time if needed. + ++-------------------------------+---------------------------------------+----------------------+ +| Tag | Description | Contact | ++===============================+=======================================+======================+ +| api_ | A bug affecting the API layer | Salvatore Orlando | ++-------------------------------+---------------------------------------+----------------------+ +| baremetal_ | A bug affecting Ironic support | Sukhdev Kapur | ++-------------------------------+---------------------------------------+----------------------+ +| db_ | A bug affecting the DB layer | Henry Gessau | ++-------------------------------+---------------------------------------+----------------------+ +| dns_ | A bug affecting DNS integration | Miguel Lavalle | ++-------------------------------+---------------------------------------+----------------------+ +| doc_ | A bug affecting in-tree doc | Edgar Magana | ++-------------------------------+---------------------------------------+----------------------+ +| fullstack_ | A bug in the fullstack subtree | Assaf Muller | ++-------------------------------+---------------------------------------+----------------------+ +| functional-tests_ | A bug in the functional tests subtree | Assaf Muller | ++-------------------------------+---------------------------------------+----------------------+ +| fwaas_ | A bug affecting neutron-fwass | Sean Collins | ++-------------------------------+---------------------------------------+----------------------+ +| gate-failure_ | A bug affecting gate stability | Armando Migliaccio | ++-------------------------------+---------------------------------------+----------------------+ +| ipv6_ | A bug affecting IPv6 support | Henry Gessau | ++-------------------------------+---------------------------------------+----------------------+ +| l2-pop_ | A bug in L2 Population mech driver | Kevin Benton | ++-------------------------------+---------------------------------------+----------------------+ +| l3-dvr-backlog_ | A bug affecting distributed routing | Ryan Moats | ++-------------------------------+---------------------------------------+----------------------+ +| l3-ha_ | A bug affecting L3 HA (vrrp) | Assaf Muller | ++-------------------------------+---------------------------------------+----------------------+ +| l3-ipam-dhcp_ | A bug affecting L3/DHCP/metadata | Miguel Lavalle | ++-------------------------------+---------------------------------------+----------------------+ +| lbaas_ | A bug affecting neutron-lbaas | Brandon Logan | ++-------------------------------+---------------------------------------+----------------------+ +| linuxbridge_ | A bug affecting ML2/linuxbridge | Sean Collins | ++-------------------------------+---------------------------------------+----------------------+ +| loadimpact_ | Performance penalty/improvements | Ryan Moats | ++-------------------------------+---------------------------------------+----------------------+ +| logging_ | An issue with logging guidelines | Matt Riedemann | ++-------------------------------+---------------------------------------+----------------------+ +| low-hanging-fruit_ | Starter bugs for new contributors | N/A | ++-------------------------------+---------------------------------------+----------------------+ +| metering_ | A bug affecting the metering layer | ? | ++-------------------------------+---------------------------------------+----------------------+ +| needs-attention_ | A bug that needs further screening | PTL/Bug Deputy | ++-------------------------------+---------------------------------------+----------------------+ +| oslo_ | An interop/cross-project issue | Ihar Hrachyshka | ++-------------------------------+---------------------------------------+----------------------+ +| ovs_ | A bug affecting ML2/OVS | Kevin Benton | ++-------------------------------+---------------------------------------+----------------------+ +| ovs-lib_ | A bug affecting OVS Lib | Terry Wilson | ++-------------------------------+---------------------------------------+----------------------+ +| qos_ | A bug affecting ML2/QoS | Miguel Ajo | ++-------------------------------+---------------------------------------+----------------------+ +| released-neutronclient_ | A bug affecting released clients | Kyle Mestery | ++-------------------------------+---------------------------------------+----------------------+ +| release-subproject_ | A request to release a subproject | Kyle Mestery | ++-------------------------------+---------------------------------------+----------------------+ +| rfe_ | Feature enhancements | Drivers Team | ++-------------------------------+---------------------------------------+----------------------+ +| sg-fw_ | A bug affecting security groups | Kevin Benton | ++-------------------------------+---------------------------------------+----------------------+ +| sriov-pci-pt_ | A bug affecting Sriov/PCI PassThrough | Moshe Levi | ++-------------------------------+---------------------------------------+----------------------+ +| unittest_ | A bug affecting the unit test subtree | Cedric Brandily | ++-------------------------------+---------------------------------------+----------------------+ +| usability_ | UX, interoperability, feature parity | PTL/Drivers Team | ++-------------------------------+---------------------------------------+----------------------+ +| vpnaas_ | A bug affecting neutron-vpnaas | Paul Michali | ++-------------------------------+---------------------------------------+----------------------+ +| xxx-backport-potential_ | Cherry-pick request for stable team | Ihar Hrachyshka | ++-------------------------------+---------------------------------------+----------------------+ + +.. _api: + +API ++++ + +* `API - All bugs `_ +* `API - In progress `_ + +.. _baremetal: + +Baremetal ++++++++++ + +* `Baremetal - All bugs `_ +* `Baremetal - In progress `_ + +.. _db: + +DB +++ + +* `DB - All bugs `_ +* `DB - In progress `_ + +.. _dns: + +DNS ++++ + +* `DNS - All bugs `_ +* `DNS - In progress `_ + +.. _doc: + +DOC ++++ + +* `DOC - All bugs `_ +* `DOC - In progress `_ + +.. _fullstack: + +Fullstack ++++++++++ + +* `Fullstack - All bugs `_ +* `Fullstack - In progress `_ + +.. _functional-tests: + +Functional Tests +++++++++++++++++ + +* `Functional tests - All bugs `_ +* `Functional tests - In progress `_ + +.. _fwaas: + +FWAAS ++++++ + +* `FWaaS - All bugs `_ +* `FWaaS - In progress `_ + +.. _gate-failure: + +Gate Failure +++++++++++++ + +* `Gate failure - All bugs `_ +* `Gate failure - In progress `_ + +.. _ipv6: + +IPV6 +++++ + +* `IPv6 - All bugs `_ +* `IPv6 - In progress `_ + +.. _l2-pop: + +L2 Population ++++++++++++++ + +* `L2 Pop - All bugs `_ +* `L2 Pop - In progress `_ + +.. _l3-dvr-backlog: + +L3 DVR Backlog +++++++++++++++ + +* `L3 DVR - All bugs `_ +* `L3 DVR - In progress `_ + +.. _l3-ha: + +L3 HA ++++++ + +* `L3 HA - All bugs `_ +* `L3 HA - In progress `_ + +.. _l3-ipam-dhcp: + +L3 IPAM DHCP +++++++++++++ + +* `L3 IPAM DHCP - All bugs `_ +* `L3 IPAM DHCP - In progress `_ + +.. _lbaas: + +LBAAS ++++++ + +* `LBaaS - All bugs `_ +* `LBaaS - In progress `_ + +.. _linuxbridge: + +LinuxBridge ++++++++++++ + +* `LinuxBridge - All bugs `_ +* `LinuxBridge - In progress `_ + +.. _loadimpact: + +Load Impact ++++++++++++ + +* `Load Impact - All bugs `_ +* `Load Impact - In progress `_ + +.. _logging: + +Logging ++++++++ + +* `Logging - All bugs `_ +* `Logging - In progress `_ + +.. _low-hanging-fruit: + +Low hanging fruit ++++++++++++++++++ + +* `Low hanging fruit - All bugs `_ +* `Low hanging fruit - In progress `_ + +.. _metering: + +Metering +++++++++ + +* `Metering - All bugs `_ +* `Metering - In progress `_ + +.. _needs-attention: + +Needs Attention ++++++++++++++++ + +* `Needs Attention - All bugs `_ + +.. _oslo: + +OSLO +++++ + +* `Oslo - All bugs `_ +* `Oslo - In progress `_ + +.. _ovs: + +OVS ++++ + +* `OVS - All bugs `_ +* `OVS - In progress `_ + +.. _ovs-lib: + +OVS Lib ++++++++ + +* `OVS Lib - All bugs `_ +* `OVS Lib - In progress `_ + +.. _qos: + +QoS ++++ + +* `QoS - All bugs `_ +* `QoS - In progress `_ + +.. _released-neutronclient: + +Released Neutron Client ++++++++++++++++++++++++ + +* `Released Neutron Client - All bugs `_ +* `Released Neutron Client - In progress `_ + +.. _release-subproject: + +Release Subproject +++++++++++++++++++ + +* `Release Subproject - All bugs `_ +* `Release Subproject - In progress `_ + +.. _rfe: + +RFE ++++ + +* `RFE - All bugs `_ +* `RFE - In progress `_ + +.. _sriov-pci-pt: + +SRIOV-PCI PASSTHROUGH ++++++++++++++++++++++ + +* `SRIOV/PCI-PT - All bugs `_ +* `SRIOV/PCI-PT - In progress `_ + +.. _sg-fw: + +SG-FW ++++++ + +* `Security groups - All bugs `_ +* `Security groups - In progress `_ + +.. _unittest: + +Unit test ++++++++++ + +* `Unit test - All bugs `_ +* `Unit test - In progress `_ + +.. _usability: + +Usability ++++++++++ + +* `UX - All bugs `_ +* `UX - In progress `_ + +.. _vpnaas: + +VPNAAS +++++++ + +* `VPNaaS - All bugs `_ +* `VPNaaS - In progress `_ + +.. _xxx-backport-potential: + +Backport/RC potential ++++++++++++++++++++++ + +* `All Liberty bugs `_ +* `All Kilo bugs `_ +* `All Juno bugs `_