This comment form is no longer interactive because the comment period is closed.

Periodic Review Standing Review Team - Standards Grading

Description:

Start Date: 06/19/2017
End Date: 08/02/2017

Associated Ballots:

Ballot Name Project Standard Pool Open Pool Close Voting Start Voting End

Filter:

Hot Answers

ERCOT joins the comments of the IRC SRC and provides the following additional comments:

Notwithstanding ERCOT’s concern with the grading methodology (please see response to Question #5), ERCOT provides the following response to the above grading question, as well as additional critique of the standard that is not addressed by the grading criteria.

ERCOT believes that there are a number of content and quality issues with TPL-001-4 and notes the following:

  • R1 seems to be redundant with MOD-032-1

  • R1.1.6 is not clear.  For example, what does “supply or demand side” mean?  Does “supply” refer only to units dispatched to meet the load?  Does “demand side” refer to demand response?  Does the “demand side” load need to be explicitly modeled?

  • The requirements and performance criteria related to R2.1.4 and R2.3.4 are not clear.  For example, should all contingencies and extreme events be considered for these sensitivities?  Should the sensitivity meet the same requirements as in Table 1?  How does an entity define a “sufficient amount to stress the System?”

  • In R2.1.5, studying events P0 and P1 is redundant with studying P6 events in R3.1.  Also, this requirement appears to meet paragraph 81 requirements because there are not any reliability based requirements to do anything with the information gained from this study

  • R2.7.4 appears to be administrative in nature (paragraph 81) or just plain not needed.  If the idea is to see if a previously identified CAP is still needed, this may be good practice, but there is not a reliability need to perform this study.  If the idea is that a previous CAP may no longer be sufficient then testing the models in the next Annual Planning Assessment will identify any reliability issues and the CAP will be modified or a new CAP will be added

  • Parts of R2 are redundant with and contain circular references to R3 and R4.  ERCOT recommends consolidating these requirements

Elizabeth Axson, On Behalf of: Elizabeth Axson, , Segments 2

- 0 - 0

TPL-001-4 R4, column Y (Comment/Rationale) – What constitutes a unit pulling out of sync is well known by power system stability engineers.  The words in R4 are clear.

Southern Company, Segment(s) 1, 6, 3, 5, 9/1/2016

- 0 - 0

Other Answers

R4.1.1: SPS term needs to be replaced with RAS.

Daniel Grinkevich, On Behalf of: Daniel Grinkevich, , Segments 1, 3, 5, 6

- 0 - 0

Ashley Stringer, On Behalf of: Oklahoma Municipal Power Authority, , Segments 1, 4

- 0 - 0

BPA believes TPL-001-4 is clear and understandable.  TPL-001-5 will be out for industry comment very soon. BPA is uncertain how valuable the grading is when a new standard is already underway.

Aaron Cavanaugh, On Behalf of: Bonneville Power Administration, WECC, Segments 1, 3, 5, 6

- 0 - 0

Comment for consideration, related to clarification of the TPL-001-4 Standard:

Requirement 4.1 states that “Studies shall be performed for planning events to determine whether the BES meets the performance requirements in Table 1…..”

Immediately after 4.1, sub-requirements 4.1.1 through 4.1.3 specify specific performance requirements. These specific performance requirements, pertaining to system stability or generator stability, are not mentioned in Table 1.

This apparent discrepancy is confusing.

Recommendation is to add sub-requirements 4.1.1 through 4.1.3 to Table 1, so that Table 1 contains all performance requirements.

 

Comment for consideration, related to clarification of the TPL-001-4 Standard:

Regarding Table 1, if the performance requirements (steady state / stability) are not being met, AND, if Table 1 indicates that non-consequential load loss and interruption of Firm Transmission Service are allowed, is a specific corrective action plan required as per Requirement 2.7 (assuming that non-consequential load loss and/or interruption of Firm Transmission Service would allow for meeting the performance requirements)? This is an example scenario where Footnote 12 does not apply.

Recommendation is to clarify within the standard whether or not a specific corrective action plan is required to be documented, as per Requirement 2.7, in the Planning Assessment for a scenario where permissible non-consequential load loss / interruption of Firm Transmission Service helps to fully meet the performance requirements. 

Robert Ganley, On Behalf of: Robert Ganley, , Segments 1

- 0 - 0

The inclusion of reference to a data collection standard (e.g. MOD-010, MOD-012, MOD-032) is unnecessary. It would be better to edit R1 to remove direct reference and obligations to another standard.

An appropriate definition of “proposed material generation additions” will vary from Interconnection to Interconnection and entity to entity. Language should be added to the Guidelines and Technical Basis that directs each entity to determine an appropriate definition for itself based on engineering judgement and technical rationale and document them.

The terms “study”, “analysis”, and “assessment” are used throughout the standard. These terms are different and are used properly in the standard. However, language could be added to the Guidelines and Technical Basis to describe each of the terms and clarify how they differ.

Replace “SPS” with “RAS”. It is preferable to replace the term “SPS” with “RAS”, but not necessary because the terms are synonymous.

It is unclear whether the PC or TP criteria takes precedence over each other. MISO has PC criteria which are applicable, unless a TP has criteria that is specific to it area. This precedence arrangement is working. It should not be assumed that PC regional criteria will by default be more appropriate than PC local criteria.  

The term “Methodology” in R6 is unclear. A description of the term could be added to the Guidelines and Technical Basis. If the development of the description reveals that a better term or wording should be used, then the requirement language should be revised.

“In conjunction with” in R7 in unclear. MISO has engaged it TP in the development of the requirement responsibilities. This approach to be compliant with R7 has been reasonable and appropriate. A description of this phase could be added to the Guidelines and Technical Basis for those are unclear about its meaning.

Lauren Price, On Behalf of: Lauren Price, , Segments 1

- 0 - 0

no comment

sean erickson, On Behalf of: Western Area Power Administration, , Segments 1, 6

- 0 - 0

These are comments from the ISO/RTO Council (IRC) Standard Review Committee (SRC). The following organizations concur with the comments listed below:

PJM, ISO-NE, MISO, IESO, ERCOT, CAISO, NYISO

SPP does not sign on to the following comments:

TPL-001-4

[Given the SRT’s question itself and in an effort to aid the SRT in the development of CONTENT scores in the current process] The SRC offers the following comments:

Where possible requirements should use objective measureable terms. TPL-001-4 uses several subjective terms, e.g.  R1 includes such undefined terms as “shall maintain”, and “consistent with”. The intent behind those terms are conceptually clear and well understood but those terms themselves are not quantifiable thereby both leaving a gap in what is intended and leaving enforcement and compliance as a subjective matter.

R1. Each Transmission Planner and Planning Coordinator shall maintain System models within its respective area for performing the studies needed to complete its Planning Assessment. The models shall use data consistent with that provided in accordance with the MOD-010 and MOD-012 standards, supplemented by other sources as needed, including items represented in the Corrective Action Plan, and shall represent projected System conditions. This establishes Category P0 as the normal System condition in Table 1

1.1. System models shall represent:

1.1.1. Existing Facilities

1.1.2. Known outage(s) of generation or Transmission Facility(ies) with a duration of at least six months.

1.1.3. New planned Facilities and changes to existing Facilities

1.1.4. Real and reactive Load forecasts

1.1.5. Known commitments for Firm Transmission Service and Interchange

1.1.6. Resources (supply or demand side) required for Load

If the intent is that both PC and TPs use the same data then that should be explicitly stated. If that is the case then the PC should be assigned the final say in disputed cases.

Planning Assessments include both Planning Studies and Corrective Action Plans. Corrective Action Plans are potential areas of conflict. In the current requirements if a TP’s CAP addresses its assessed problem, and the PC CAP addresses its issue, then both are compliant. However, if the PC’s CAP requires the TP to do something different from the TPS CAP there is no recognition of who has the authority over the final set of CAPS.

Please see additional Comments under Question #5 below for the SRC’s overall position on Standards Grading.

IRC-SRC, Segment(s) 2, 8/2/2017

- 0 - 0

It is unclear how a short term Corrective Action Plan could be developed for Contingency analysis as outlined in TPL-001-4 Requirement R4, Part 4.3.  Requirement R2, Part 2.7 states, “For planning events shown in Table 1, when the [stability] analysis indicates an inability of the System to meet the performance requirements in Table 1, the Planning Assessment shall include Corrective Action Plan(s) addressing how the performance requirements will be met.  Revisions to the Corrective Action Plan(s) are allowed in subsequent Planning Assessments but the planned System shall continue to meet the performance requirements in Table 1.”  Under Table 1, non-consequential load loss is not allowed for many of the planning events.  Even where it is allowed, non-consequential load loss cannot be achieved fast enough to meet the standard’s performance requirements for contingencies studied in Part 4.3; i.e., there is no way that the operator can take the necessary action to drop the load in under a second.  Since no other options are provided for compliance in the standard besides load drop (for some planning events) and since projects cannot be built instantaneously, a compliance violation seems to exist as soon as an entity performs the Stability study that is required by TPL-001-4 Requirement R4, Part 4.3.

PPL NERC Registered Affiliates, Segment(s) 3, 1, 5, 6, 2/9/2017

- 0 - 0

We agree with APPA's comments:

 TPL-001-4 is not clear or well understood.  Therefore, the standard should be included in the Periodic Reviews.

FMPA, Segment(s) , 8/2/2017

- 0 - 0

Daniel Gacek, On Behalf of: Exelon, , Segments 1, 3, 5, 6

- 0 - 0

Illinois Municipal Electric Agency agrees with comments submitted by American Public Power Association.  

Mary Ann Todd, On Behalf of: Illinois Municipal Electric Agency, , Segments 4

- 0 - 0

Requirement R7 is needed. From our perspective, the Planning Coordinators (PCs) that have several Transmission Planners (TPs) in their PC area have the need for a roles and responsibilities requirement to ensure comparability of expectations among TPs.

SPP Standards Review Group, Segment(s) , 8/2/2017

- 0 - 0

Duke Energy , Segment(s) 1, 5, 6, 4/10/2014

- 0 - 0

  1. We thank the commenters representing the Regional Entities for identifying discrepancies within the NERC Reliability Standard TPL-001-4.  The purpose of the standard is to establish Transmission system planning performance requirements for the planning horizon to produce a Planning Assessment.  However, the standard does not utilize a uniform reference for those activities conducted to produce a “documented evaluation.”  We believe the standard should clearly identify the requirements necessary to conduct an action, such as “perform an evaluation,” and reflect such actions within the NERC Glossary of Terms similar to other terms (i.e. Real-time Assessment and Operational Planning Analysis).
  2. We believe the standard’s content should be clarified further, as the standard currently references retired NERC Reliability Standards (e.g. Requirement R1), terminology (e.g. Special Protection Systems), and interpretations (e.g. sufficient, credible, material changes, acceptable, reliability-related need, functional entity, 30 days, etc.).  The standard jumps from one subpart to the next (e.g. studies are performed to assess a list of extreme events identified in another subpart that are based on Table 1) and places unnecessary compliance criteria on requirements that simply capture the evaluation results of Planning Assessments and supportive modeling maintenance.  For example, we believe conducting a contingency analysis, as part of the steady state portion of a Planning Assessment, should have its own requirement and not identify a list of devices that could be included in the simulation.  Likewise, the language within Requirement R6 uses redundant language to require TPs and PCs to document a methodology to identify System instability conditions within their Planning Assessment.  The standard also has requirements with identifiable P81 criteria, such as identifying individual responsibilities with a Planning Coordinator, in conjunction with their Transmission Planners, and distributing Planning Assessment within specific timelines.

ACES Standards Collaborators, Segment(s) 1, 3, 5, 4, 8/2/2017

- 0 - 0

We generally support the comments made for TPL-001-4 in the RE tab.  Additionally, we believe R7 should be retained, but should be the first requirement of the Standard.

 

The standard needs to further address an existing discrepancy and provide clarification as follows:

·         Use of non-consequential load loss:  In the Table 1 included in the standard, the use of non-consequential load loss is allowed under Footnote 12 conditions for P1, P2, and P3 planning events for elements operated at EHV level. However, the planning events P4 and P5, which would include the same elements as in P1, P2 & P3, do not allow the use of non-consequential load loss at EHV level.

 

Regarding Table 1, if the performance requirements (steady state / stability) are not being met, and if Table 1 indicates that non-consequential load loss and interruption of Firm Transmission Service are allowed. Is a specific corrective action plan required as per Requirement 2.7 (assuming that non-consequential load loss and/or interruption of Firm Transmission Service would allow for meeting the performance requirements)? This is an example scenario where Footnote 12 does not apply. Recommendation is to clarify within the standard whether or not a specific corrective action plan is required to be documented, as per Requirement 2.7, in the Planning Assessment for this scenario

 

·         Location of the fault while “component failure of a Protection System” is studied:  Generally, in the planning studies the faults are applied on the buses since they produce the more severe system impacts. However, when the “component failure of a Protection System” is considered, a bus fault or a close in fault may still be cleared remotely by using back up protections (remote 21 timed, 51, 51N etc.…).  However, when the fault location is moved along the circuits there may be locations on some of the circuits, where the faults will remain un-cleared, since the remote back up protection systems may not be able to detect it.

·         Applying P5 criteria to certain non- BES elements connected to BES buses (e.g., radial circuits supplying loads). The standard needs to clarify which protection systems are subject to it since an un-cleared close in fault on a non-BES element connected to a BES bus has the same consequence as an un-cleared close in fault on a BES element. Do the protection systems installed on non-BES elements but connected to BES buses and therefore providing protection to the BES bus (the stub bus portion of the non-BES element is part of the BES bus) need to meet redundancy criteria of P5?

An alternative the SRT can consider to revising the standard is to provide a guideline that addresses the discrepancy and provides clarification.

 

Requirement 4.1 states, “Studies shall be performed for planning events to determine whether the BES meets the performance requirements in Table 1…..” Immediately after 4.1, sub-requirements 4.1.1 through 4.1.3 specify specific performance requirements, which are not mentioned in Table 1. This apparent discrepancy is confusing. Recommendation is to add sub-requirements 4.1.1 through 4.1.3 to Table 1.

RSC, Segment(s) 1, 0, 2, 4, 5, 6, 7, 3, 7/18/2017

- 0 - 0

OPG supports the TPL-001-4 RE tab comments. We are of the opinion that what the SRT should consider in developing its final content scores on this standard requirements is not paramount however the following can be considered: SRT member comments, discrepancies identified via the comments provided during this review of the Standards Grading as well as the currently identified existing needs for revision (i.e. SAR/TPL-001-5 currently awaiting BOD adoption).

As a general rule the lack of consensus is a clear indication that a standard should be prioritized for Periodic Review.

 

OPG is of the opinion that there are aspects of the standard’s content that are not clear or well-understood.

It is not clear if Table 1 Steady State & Stability Performance Planning Events/Extreme Events covers the Power system Geomagnetic Disturbances (GMD) events category.

Steady State & Stability Performance Planning Events category P5 requires additional clarification regarding the actual location of the applied fault. Are there any consideration for the cases where the fault is not detected by the back-up protection? Are there any cases where consideration should be given to fault location on the CT/PT secondary circuits? Is there any category for the cases where a fault (located on BES or Non-BES element) can trigger tripping of the non-affected BES Elements due to protective relaying inadequate phasors?

Should the SRT consider an event category where there are two poles of the stuck circuit breakers remaining closed?

David Ramkalawan, On Behalf of: David Ramkalawan, , Segments 5

- 0 - 0

Rick Applegate, On Behalf of: Rick Applegate, , Segments 1, 3, 4, 5, 6

- 0 - 0

APPA believes that the standard is not clear or well understood.  Therefore, the standard should be included in the Periodic Reviews.

Jack Cashin, On Behalf of: American Public Power Association, , Segments 3, 4

- 0 - 0

R1: We believe the requirement needs to be updated to reflect that MOD-010/12 are retired.

R2: We agree with the comments on the RE tab.

R8: We agree with the comments on the RE tab. 

Sergio Banuelos, On Behalf of: Tri-State G and T Association, Inc., MRO, WECC, Segments 1, 3, 5

- 0 - 0

Hot Answers

ERCOT does not join the comments of the IRC SRC in response to Question #2, and instead provides the following comment:

Notwithstanding ERCOT’s concern with the grading methodology (please see response to Question #5), ERCOT provides the following response:

In MOD-032-1, the phrase “jointly develop” replaced the phrase “in conjunction with,” which is still used in many standards.  Both phrases are confusing because neither “jointly” nor “in conjunction with” is defined, and therefore is ambiguous from both an entity and an auditor standpoint.  For example:

  1. Does every TO in the PC’s area have to participate or just a few? 

  2. What constitutes participation?  Does attending a meeting or being a member of the relevant stakeholder working group constitute participation?  If an entity approves the working group’s procedures and deliverables at a higher-level stakeholder meeting, does this count as “jointly developing”?

    ERCOT believes that NERC reliability standards should generally avoid assigning joint responsibility for specific duties and should instead clearly delineate the obligations of each responsible entity.  Assigning joint responsibility will often result in those parties allocating responsibilities between themselves based on their relative bargaining power instead of an objective analysis of which party is more optimally positioned to perform the required function. 

  3.  

Elizabeth Axson, On Behalf of: Elizabeth Axson, , Segments 2

- 0 - 0

Southern Company, Segment(s) 1, 6, 3, 5, 9/1/2016

- 0 - 0

Other Answers

In the NY Control Area the term “jointly develop” is well understood and documented.  The PC and TPs have a coordinated functional registration for all TP requirements.

Daniel Grinkevich, On Behalf of: Daniel Grinkevich, , Segments 1, 3, 5, 6

- 0 - 0

Ashley Stringer, On Behalf of: Oklahoma Municipal Power Authority, , Segments 1, 4

- 0 - 0

BPA believes as long as we are coordinating with the TPs in our PC area, then the R1 process is “jointly developed."

Aaron Cavanaugh, On Behalf of: Bonneville Power Administration, WECC, Segments 1, 3, 5, 6

- 0 - 0

Robert Ganley, On Behalf of: Robert Ganley, , Segments 1

- 0 - 0

It is unclear whether “jointly develop” means. A description of the term could be added to the Guidelines and Technical Basis, such as each PC and each TP must: (1) provide input for creating the PC area’s data requirements and reporting procedures, (2) participate directly in the creation of the joint requirements and procedures documents, (3) declare acceptance, or acceptance with specific modifications to, the proposed requirements and procedures, and (4) have evidence that it performed of items 1, 2, and 3. It might be clearer and simpler to obligate the PC to develop the PC area’s data requirements and reporting procedures, in conjunction with the PC’s TPs, using the language of TPL-001-4_R7.

Lauren Price, On Behalf of: Lauren Price, , Segments 1

- 0 - 0

sean erickson, On Behalf of: Western Area Power Administration, , Segments 1, 6

- 0 - 0

The following organizations concur with the comments listed below:

PJM, ISO-NE, MISO, IESO, CAISO, NYISO

SPP and ERCOT do not concur with the following comments:

R1. Each Planning Coordinator and each of its Transmission Planners shall jointly develop steady-state, dynamics, and short circuit modeling data requirements and reporting procedures for the Planning Coordinator’s planning area that include:

The concept of joint-development is a clear concept and it is recognized as being of value to the planning process.

However, the concept of joint-development is not well suited to mandatory reliability standards that depend on a well-defined establishment of responsibilities and reliability providers of last resort. With joint-development there is no quantitative way to identify “jointness” as it applies to authority and responsibility as well as to compliance.

R1 applies to both PCs and TPs. There currently is a well-defined hierarchy and difference between a given PC and TPs in its (the PC’s) area.

  • PC: (The responsible entity that coordinates and integrates transmission Facilities and service plans, resource plans, and Protection Systems.)

  • TP: (The entity that develops a long-term (generally one year and beyond) plan for the reliability (adequacy) of the interconnected bulk electric transmission systems within its portion of the Planning Authority area.)

The reliability disconnect is that TPs have a much narrower scope and obligation and depend upon their PCs to access/address the wider area relationships and impacts. In short the Planning Assessments and models are different between PCs and TPs, and to address the two assessments and models in the same requirements suggests similarities that are not there.

R2. Each Balancing Authority, Generator Owner, Load Serving Entity, Resource Planner, Transmission Owner, and Transmission Service Provider shall provide steady-state, dynamics, and short circuit modeling data to its Transmission Planner(s) and Planning Coordinator(s) according to the data requirements and reporting procedures developed by its Planning Coordinator and Transmission Planner in Requirement R1. For data that has not changed since the last submission, a written confirmation that the data has not changed is sufficient.

In light of R2, R1 seems to be unnecessary. R1 seems to be predicated on the notion of a seamless connection between a PC and the TPs in its area. R1 does not account for overlapping TPs nor does it account for incompatible study results. R1 does not provide for a provider of last resort in the case in which there is no agreement on the scope and assessment of work. For a PC that does not agree with one of its TPs on the need for data or reporting requirements, whose requirements are enforced? R1 is silent on this. “Jointness” is an outcome not a defined process.

Moreover, the requirement can be read to require that each TP have a set of requirements with each and every other TP in the PC area (even though the TPs have no way of knowing who the other TPs are).

Please see additional Comments under Question #5 below for the SRC’s overall position on Standards Grading.

IRC-SRC, Segment(s) 2, 8/2/2017

- 0 - 0

PPL NERC Registered Affiliates, Segment(s) 3, 1, 5, 6, 2/9/2017

- 0 - 0

We agree with APPA's comments:

There should be a clear definition of responsibilities instead of just “jointly develop”.

Feedback from some entities indicates that the PC and TP do not have jointly developed requirements which is potentially leading to duplication of effort.      

FMPA, Segment(s) , 8/2/2017

- 0 - 0

Exelon agrees with other commenters that "jointly develop" should be clarified.

Daniel Gacek, On Behalf of: Exelon, , Segments 1, 3, 5, 6

- 0 - 0

Illinois Municipal Electric Agency agrees with comments submitted by American Public Power Association.  

Mary Ann Todd, On Behalf of: Illinois Municipal Electric Agency, , Segments 4

- 0 - 0

SPP Standards Review Group, Segment(s) , 8/2/2017

- 0 - 0

Duke Energy , Segment(s) 1, 5, 6, 4/10/2014

- 0 - 0

The purpose of this standard is to establish consistent modeling data requirements and reporting procedures for the development of planning horizon studies.  However, we observe many PCs and their TPs attaching meeting minutes and coordination e-mails with their specification to demonstrate a joint development.  It is the responsibility of the PC to coordinate Planning Assessments within their planning area, and that infers a responsibility that the PC should also provide a defined modeling specification.  We have seen many PCs draft their specifications through a stakeholder process with their TPs in order to achieve modeling efficiencies within the condensed timelines.  We also believe that any activity identified to clarify this standard should include the removal of the Load Serving Entity and Planning Authority functions from associated requirements.

ACES Standards Collaborators, Segment(s) 1, 3, 5, 4, 8/2/2017

- 0 - 0

The language is well understood.  However, we would like the SRT to consider the following changes should the standard be considered for review:       

 

(1)    Applicable to R2 - we would like to see the removal of the need for each balancing authority, generator owner, load serving entity, resource planner, transmission owner and transmission service provider etc. annually reporting to the PC if data has not changed since last submission (this is an administrative overburden on all parties which adds no value) and report/submit data only if data has changed.

 

(2)    Applicable to R4 - Removal of the need to submit short circuit data to the Electric Reliability Organization (ERO) or its designee to support creation of the Interconnection-wide short circuit cases (no such cases are created) or needed.

RSC, Segment(s) 1, 0, 2, 4, 5, 6, 7, 3, 7/18/2017

- 0 - 0

OPG is of the opinion that further clarification of “jointly develop” is required i.e.: common participation, coordination and approval. Lack of clarity can trigger potential compliance evidentiary issues.

David Ramkalawan, On Behalf of: David Ramkalawan, , Segments 5

- 0 - 0

The phrase “jointly develop” is easily understood to mean the parties will work together to determine, specify, and document the “modeling data requirements and reporting procedures” as further described in Requirement 1.

However, MOD 32-R1 has increased risk to the WECC interconnection because of confusion as to who should develop modeling requirements.  Prior to MOD-032, WECC developed interconnection wide modeling guidelines with input from all Functional Entities.  Since MOD-032 directs individual PCs to develop their own modeling requirements, there are now more than 20 different sets of modeling requirements throughout WECC.

Rick Applegate, On Behalf of: Rick Applegate, , Segments 1, 3, 4, 5, 6

- 0 - 0

There should be a clear definition of responsibilities instead of just “jointly develop”.

Feedback from some entities indicates that the PC and TP do not have jointly developed requirements which is potentially leading to duplication of effort.      

 

Jack Cashin, On Behalf of: American Public Power Association, , Segments 3, 4

- 0 - 0

Yes, Tri-State believes the language accomplishes the reliability task while still providing flexibility for the industry to accomplish it as they prefer. Each entity in each region will approach this requirement uniquely but the collaboration between the PC and the TP will occur with the requirement as written. 

Sergio Banuelos, On Behalf of: Tri-State G and T Association, Inc., MRO, WECC, Segments 1, 3, 5

- 0 - 0

Hot Answers

ERCOT joins the comments of the IRC SRC and provides the following additional comments:

Notwithstanding ERCOT’s concern with the grading methodology (please see response to Question #5), ERCOT provides the following response on FAC-001-2.

FAC-001-2 appears to meet the P81 criteria of being administrative in nature with little or no reliability impact since it imposes documentation requirements without any requirement to actually use that documentation.  Hence, it does not seem to meet any particular reliability objective. This standard qualifies under Paragraph 81 criteria as a “documentation standard” and should be reviewed by an EPR team under the Paragraph 81 test, and retired.

Elizabeth Axson, On Behalf of: Elizabeth Axson, , Segments 2

- 0 - 0

Southern Company, Segment(s) 1, 6, 3, 5, 9/1/2016

- 0 - 0

Other Answers

Daniel Grinkevich, On Behalf of: Daniel Grinkevich, , Segments 1, 3, 5, 6

- 0 - 0

Ashley Stringer, On Behalf of: Oklahoma Municipal Power Authority, , Segments 1, 4

- 0 - 0

BPA believes that the FAC-001-2 Requirement R3 does not cause a reliability concern; however BPA believes FAC-001-2 Requirement R3 is confusing because R3 is procedural in nature. Coordination with affected systems is documented in other study procedures such as the Large Generation Interconnection process included in utility tariffs. The purpose of FAC-001 is for the Transmission Owner to document and make Facility interconnection requirements available so that entities seeking to interconnect will have the necessary information.  BPA believes that the R1 requirement fulfills this purpose.

BPA is not registered as a Generator Owner. R2 and R4 requirements are not applicable to BPA because they are for Generator Owners.

Aaron Cavanaugh, On Behalf of: Bonneville Power Administration, WECC, Segments 1, 3, 5, 6

- 0 - 0

Concerning FAC-001-2, R3, R3.1:

There is some apparent overlap between R3.1 and FAC-002-2 Req. # 3 and 4 that could be a concern during an audit.

FAC-001-2 R3.1 mentions addressing “Procedures for coordinated studies of new or materially modified existing interconnections and their impacts on affected system.”

 

FAC-002-2 R3 and R4 essentially mention “Each Transmission Owner…… shall coordinate and cooperate on studies with its Transmission Planner or Planning Coordinator.” R1 mentions that the “Transmission Planner and each Planning Coordinator shall study….”

 

This apparent overlap with respect to actions to take regarding studies is confusing / ambiguous such that it would create a concern between an auditor and an entity. For example, FAC-002-2 R3 and R4 mention coordination and cooperation on studies as actions for the Transmission Owner, while FAC-001-2 R3.1 mentions the documentation of procedures for coordinated studies as actions for the Transmission Owner. It could be interpreted that the Transmission Planner or Planning Coordinator should be the applicable entity for FAC-001-2 R3.1, since those entities are required to perform studies as per FAC-002-2 R1.

 

Robert Ganley, On Behalf of: Robert Ganley, , Segments 1

- 0 - 0

R3 - “Materially modified” is ambiguous. As noted in in Guidelines and Technical Basis the definition will vary from entity to entity and therefore should be determined by each entity based on engineering judgement and technical rationale.

R3 & R4 – “Coordinated studies” is ambiguous. This term is not necessarily ambiguous because a single “coordinated study” definition would not describe appropriate coordination for the wide range of proper studies for different proposed interconnections. Language could be added to the Guidelines and Technical Basis indicating that the Facility Interconnection Study Agreement should delineate how the parties that will participate in the study are to coordinate.

Lauren Price, On Behalf of: Lauren Price, , Segments 1

- 0 - 0

sean erickson, On Behalf of: Western Area Power Administration, , Segments 1, 6

- 0 - 0

The following organizations concur with the comments listed below:

PJM, ISO-NE, MISO, IESO, CAISO, NYISO, SPP and ERCOT

R3 and R4 introduce the mandate to “address”. The phrase is not well-defined and may be better changed to “document” and then include the caveat “if they exist”:

R3. Each Transmission Owner shall address the following items in its Facility interconnection requirements:

3.1. Procedures for coordinated studies of new or materially modified existing interconnections and their impacts on affected system(s). ….

Suggested alternative:

R3. Each Transmission Owner shall document the following procedures if those procedures exist in the TOs Facility interconnection requirements:

3.1. Procedures for coordinated studies of new or materially modified existing interconnections and their impacts on affected system(s)….

Please see additional Comments under Question #5 below for the SRC’s overall position on Standards Grading.

IRC-SRC, Segment(s) 2, 8/2/2017

- 0 - 0

PPL NERC Registered Affiliates, Segment(s) 3, 1, 5, 6, 2/9/2017

- 0 - 0

No comment

FMPA, Segment(s) , 8/2/2017

- 0 - 0

While we are not aware of adverse reliability events or compliance violations related to the terms "materially modify" or "coordinate and cooperate" we agree they are ambiguous and the requirement language should be clarified.

Daniel Gacek, On Behalf of: Exelon, , Segments 1, 3, 5, 6

- 0 - 0

Illinois Municipal Electric Agency agrees with comments submitted by American Public Power Association.  

Mary Ann Todd, On Behalf of: Illinois Municipal Electric Agency, , Segments 4

- 0 - 0

The SRT should add the term “materially modified” to Requirement R4 sub-parts 4.1 and 4.2. We feel this addition to the language will provide consistency with the existing language in Requirement R3 and its sub-parts 3.1 and 3.2.

SPP Standards Review Group, Segment(s) , 8/2/2017

- 0 - 0

Requirement 2 could be worded more clearly in that it does not seem entirely clear that the GO is required to document Facility interconnection requirements regardless of whether they receive a request to provide them.

R3, subpart 3.2 and R4, subpart 4.2 could be made more clear by better defining “those responsible for the reliability of affected system(s)”.  The statement is so open ended that it may be difficult to prove all entities have been notified and is subject to differences of opinion as to who they are.

Duke Energy , Segment(s) 1, 5, 6, 4/10/2014

- 0 - 0

FAC-001-3 is currently pending approval at FERC.  We do not understand why the SRT is considering comments on this standard as a final order has not yet been issued by the Commission.  Still, we believe P81 criteria still exists in many aspects of this standard.  Furthermore, references to “materially modified” and vaguely notifying impacted entities who are responsible for the reliability of affected systems are defined within the standard’s supplement materials, but can be easily overlooked by auditors.

ACES Standards Collaborators, Segment(s) 1, 3, 5, 4, 8/2/2017

- 0 - 0

RSC, Segment(s) 1, 0, 2, 4, 5, 6, 7, 3, 7/18/2017

- 0 - 0

OPG is of the opinion that clarification is required for 'materially modified' and 'coordinate and cooperate' to avoid confusion or ambiguity.

David Ramkalawan, On Behalf of: David Ramkalawan, , Segments 5

- 0 - 0

Requirements R3.1 and R4.1 expect the identified functional entities to address the “Procedures for coordinated studies of new … interconnections and their impacts on affected system(s).”  These could be interpreted to mean that only the procedures require addressing and not the actual impacts resulting from the new interconnections.  It might be best to include separate sub-requirements that deal with any impacts on affected system(s) identified by the coordinated studies and how they will be addressed.

Also, R2 is unclear as to whether GO has to develop and maintain interconnection requirements, or whether the GO can wait until a study request is received before developing interconnection requirements.

Rick Applegate, On Behalf of: Rick Applegate, , Segments 1, 3, 4, 5, 6

- 0 - 0

The language is sufficiently clear.  

Jack Cashin, On Behalf of: American Public Power Association, , Segments 3, 4

- 0 - 0

Tri-State believes FAC-001-2 Requirement R2 is poorly written. If the intent is as we understand it we believe the following language would clarify the requirement. “Following full execution of an Agreement to conduct a study on the reliability impact of interconnecting a third party Facility to a Generator Owner’s existing Facility that is used to interconnect to the Transmission System; each Generator Owner shall document and provide the Facility interconnection requirements within 45 calendar days.

Sergio Banuelos, On Behalf of: Tri-State G and T Association, Inc., MRO, WECC, Segments 1, 3, 5

- 0 - 0

Hot Answers

ERCOT joins the comments of the IRC SRC and provides the following additional comments:

FAC-001-2

Please see ERCOT’s comments in response to Question #3.

FAC-002-2

FAC-002-2 R1 is not clear about which functional entity (TP or PC) should be performing the required study.  The requirement itself states that both the TP “and” PC should be performing the study, which would not be cost effective; however, the SDT consideration of comments makes it clear that the SDT intended only one entity (TP or PC) to perform the study, but not both.  This lack of clarity was brought up multiple times by multiple entities during the development process, but this ambiguity was still not addressed. Requirements R2, R3, R4, and R5 also imply that only one entity should perform the study.

Please see the following exchange from the Consideration of Comments:

“Several commenters asked the SDT to resolve the Planning Coordinator/Transmission Planner “and” versus “or” terminology among R1, the other requirements, and the Measures and VSLs. One commenter asked for clarification of who leads the study when the Transmission Planner and Planning Coordinator are not the same. The SDT intentionally maintained “and” in R1: “Each Transmission Planner and each Planning Coordinator.” This wording gives the Transmission Planner and the Planning Coordinator the flexibility to determine which entity will study the reliability impact, while 1.4 addresses the option for the entities to jointly study the reliability impact. Once the Transmission Planner and the Planning Coordinator have determined which entity will study the reliability impact, the other Applicable entities will coordinate and cooperate with either the Transmission Planner and the Planning Coordinator so the remaining requirements say “Transmission Planner or Planning Coordinator,” and both the Measure and the VSL language use “or.”  (found at http://www.nerc.com/pa/Stand/FAC%20FiveYear%20Review%20Team/Comment_Report_responses_06122014.pdf)”

This justification is not satisfactory as the words “and” and “or” continue to have to different meanings, and the SDT intent is not plainly understood in the context of the standard language.  In fact, one would have to look back at the Consideration of Comments explanation to understand the conflict of the “and” vs. “or” meaning and what the SDT actually intended. 

Additionally, the requirement implies that all new generation, transmission, and electricity end-user Facilities be studied.  However, it is not practical that a full set of studies, including dynamic stability studies, be performed for immaterial additions to the grid (note that the "materially modifying" qualifier only applies to existing facilities).

FAC-002, R1 and M1 also conflict.  R1 states “Each TP and each PC shall…” and M1 states “Each TP or each PC shall…”

 

Elizabeth Axson, On Behalf of: Elizabeth Axson, , Segments 2

- 0 - 0

MOD-032.  This standard is causing GOs a significant amount of work due to the wide variations in requests from multiple TPs and PCs in the various NERC Regions.  Some TPs confuse their MOD-032 data requests with the reporting requirements of MOD-025, 026 and 027 model validations standards.  MOD-032 needs to focus only on critical data that is needed for system reliability.  Much of the modeling data currently being requested from GOs can be categorized as “not essential” or “low priority” without rigid deadlines for submittals.  Also, “changes” in modeling data needs additional definition with practical allowances for equipment tolerances and parameter accuracies needed for system reliability.

Southern Company, Segment(s) 1, 6, 3, 5, 9/1/2016

- 0 - 0

Other Answers

No comment.

Daniel Grinkevich, On Behalf of: Daniel Grinkevich, , Segments 1, 3, 5, 6

- 0 - 0

COM-001  should have the highest priority for review. COM-001 in particular is administrative in nature and unnecessary. In the year 2017, no entity is doing business without a phone. There is an unecessary burden in verifying an entity has a phone to be compliant.

Ashley Stringer, On Behalf of: Oklahoma Municipal Power Authority, , Segments 1, 4

- 0 - 0

BPA offers the following comments regarding priority reviews:

  • BAL-003 should be low if not the lowest priority since it is already getting plenty of attention from NERC due to a SAR for BAL-003 that has been submitted.  BPA believes that a periodic review on top of that is redundant. 

  • COM-001 was just revised and the new version goes into effect on 10/1/2017, so this priority review should be a very low priority.

  • BPA believes that with COM-002 there are no issues that would suggest it needs to be revised. 

  • Complying with MOD-032 and TPL-001 is clear thus BPA would rate low the need for a periodic review for these two standards. 

  • As a result of the above comments FAC-001, FAC-002 and IRO-009 should be a higher priority.

Aaron Cavanaugh, On Behalf of: Bonneville Power Administration, WECC, Segments 1, 3, 5, 6

- 0 - 0

Robert Ganley, On Behalf of: Robert Ganley, , Segments 1

- 0 - 0

TPL-001-4 seems to be a possible candidate to start reviewing in 2018.

  • This standard has only been active about 2 years and a revision (5) to address FERC’s directives is already in progress. However, there are a number of issues that are noted above in the comments for Question 1 and Question 5. In addition, R2/R3/R7 have a low Quality scores and R4/R5/R6/R8 have fair Quality scores.  The consideration of revisions may take a few years, so it could be started next year.

 

FAC-001-2 seems to be a reasonable candidate to start reviewing and revising in 2019.

  • This standard has only been active about 1 year. However, R1-R4 have low Quality score and R1/R2/R3 seem to be business practices, rather than reliability requirements. So, the consideration of revisions could be started the year after next.

 

BAL-003-1, FAC-002-2, and MOD-032-1 seem to be reasonable candidates for review and revision in 2020.

  • BAL-003-1 – This standard has been active about 3 years. It has good Content and Quality scores. However, a few issues have been identified: the need to identify variable frequency bias limit and interconnection frequency definition; R3 allows regulation overlap; and in R3/R4 it is unclear whether they apply to Interconnection or area.

  • FAC-002-2 – This standard has only been active about 1 year. In addition, the standard has good Content and Quality scores, but R1 has possible PC versus TP role/responsibility ambiguity. Some comments were provided Question 3. The consideration of revisions to FAC-002-2 would be better evaluated after the standard has been in effect for a few more years.

  • MOD-032-1 – This standard has only been active about 1 year. In addition, the standard has good Content and Quality scores. However, the ERO should not be an applicable entity; LSE is no longer a registered entity; and R4 who is responsible for creating the models. Some comments were provided Question 2. The consideration of revisions to MOD-032-1 would be better evaluated after the standard has been in effect for a few more years.

 

COM-001-2, COM-002-4, and IRO-009-2 seem to be reasonable candidates for review and revision beyond 2020

  • COM-001-2 – This standard has been active for 3 years and COM-001-3 will become effective in October 2017. Although the grading activity identified several areas for improvement of COM-001-2, the consideration of improvements to COM-001-3 would be more relevant and likely after it has been in effect for a few year. [R1 has a fair Quality score. R3 has a low Quality score. Consider removing the R1/R2/R3/R5 exclusions regarding adjacent RCs with DC tie to another Interconnection. R2/R4 are very administrative and burdensome. R6-R8 same concerns as R1. R10/R11 make notification consistent with VAR-002.

  • COM-002-4 – This standard has only been active about 1 year and COM-002-5 has been approved by NERC in February 2016 and will become effective after FERC approval.   Although the grading activity identified several areas for improvement of COM-002-4, the consideration of improvements to COM-002-5 would be more relevant after it has been in effect for a few years.

Lauren Price, On Behalf of: Lauren Price, , Segments 1

- 0 - 0

BAL-003 has shown to have issues due to recent SARs

COM-002 a lot of entities have taken the 3-part communication to an extreme for all "operating instructions" which then creates a back log of work and documentation for no reliability benefit.

sean erickson, On Behalf of: Western Area Power Administration, , Segments 1, 6

- 0 - 0

No Comment

IRC-SRC, Segment(s) 2, 8/2/2017

- 0 - 0

Due to ambiguities in such terms as “case” and “model” (used interchangeably but seemingly with different meanings at times) and ambiguities regarding compliance as discussed in Question 1 above, and other ambiguities and shortcomings, the TPL-001 standards should have the highest priority for review in 2018.

PPL NERC Registered Affiliates, Segment(s) 3, 1, 5, 6, 2/9/2017

- 0 - 0

We agree with APPA's comments:

   The following in order of priority are the standards public power believes need review in 2018:

COM-001 should be considered for the new Paragraph 81 effort to be eliminated or simplified through a PR at a minimum.  Entities have communication capability as a matter of conducting regular operations and business.  COM-001 is an administratively burdensome standard with no value-added to reliability.MOD-032 has a number of issues that could be addressed. Material amounts of unnecessary data that must be provided in an unrealistic timeframe.  APPA members have found that the data requested is done so on and inconsistent basis by the Regions.  Moreover there is duplication of data and requirements with MOD-026 and MOD-27.

 

TPL-001 needs more clarity.  The grading process demonstrates this need.

 

FMPA, Segment(s) , 8/2/2017

- 0 - 0

Daniel Gacek, On Behalf of: Exelon, , Segments 1, 3, 5, 6

- 0 - 0

Illinois Municipal Electric Agency agrees with comments submitted by American Public Power Association.  

Mary Ann Todd, On Behalf of: Illinois Municipal Electric Agency, , Segments 4

- 0 - 0

The Operational Standards should take first priority. TPL-001 and BAL-003 are currently in the drafting team phase and COM-001 is effective in October 2017.

IRO-009, COM-002, MOD-032, FAC-002, BAL-003, COM-001, and TPL-001.

SPP Standards Review Group, Segment(s) , 8/2/2017

- 0 - 0

Duke Energy , Segment(s) 1, 5, 6, 4/10/2014

- 0 - 0

Unless there is a reliability impact that would require a revision to the identified Reliability Standards, we believe these standards should be allowed time to mature.  BAL-003, COM-001, and FAC-001 should have a low priority for further development, as these standards have recently been revised through other standard development activities.  We believe there are flaws with MOD-032 and TPL-001, and both standards should be given a higher priority for review in 2018.  We also identify COM-002 as a candidate due to references of the English language in oral and written Operating Instructions.

ACES Standards Collaborators, Segment(s) 1, 3, 5, 4, 8/2/2017

- 0 - 0

We believe TPL-001 should be prioritized for review in 2018 since it can benefit from various clarifications.

 

The SRT should consider the following in determining highest priority for review in 2018:  

·         BAL-003 – It seems that the process where NERC publishes the list of events for each interconnection at least quarterly so BAs can calculate performance throughout the year is delayed.  The SRT should consider whether this publishing timeframe is still feasible.

·         Whether any comments received regarding the current content of these standards can be addressed through alternative tools such as guidelines.

Whether past comments received during the balloting of these standards were not sufficiently addressed, and continue to be repeated during the standards grading process.

RSC, Segment(s) 1, 0, 2, 4, 5, 6, 7, 3, 7/18/2017

- 0 - 0

OPG is of the opinion that TPL-001-4 review should be prioritized in concordance with TPL-001-5 (currently awaiting BOD adoption). The second standard to be considered for PR review in 2018 should be determined based on the final standards grading scores.

David Ramkalawan, On Behalf of: David Ramkalawan, , Segments 5

- 0 - 0

Rick Applegate, On Behalf of: Rick Applegate, , Segments 1, 3, 4, 5, 6

- 0 - 0

The following in order of priority are the standards public power believes need review in 2018:

COM-001 should be considered for the new Paragraph 81 effort to be eliminated or simplified through a PR at a minimum.  Entities have communication capability as a matter of conducting regular operations and business.  COM-001 is an administratively burdensome standard with no value-added to reliability.MOD-032 has a number of issues that could be addressed. Material amounts of unnecessary data that must be provided in an unrealistic timeframe.  APPA members have found that the data requested is done so on and inconsistent basis by the Regions.  Moreover there is duplication of data and requirements with MOD-026 and MOD-27.

TPL-001 needs more clarity.  The grading process demonstrates this need.

Jack Cashin, On Behalf of: American Public Power Association, , Segments 3, 4

- 0 - 0

The draft 2018-2020 Reliability Standards Development Plan (RSDP) does not have any of these standards listed in the section of standards eligible for periodic review in 2018. Why is this list of standards being considering when they are not listed on the RSDP? Unless there is a specific SAR, Tri-State would like to see standards be in effect and fully implemented for at least 5 years before having them reviewed again. 

Sergio Banuelos, On Behalf of: Tri-State G and T Association, Inc., MRO, WECC, Segments 1, 3, 5

- 0 - 0

Hot Answers

ERCOT joins the comments of the SRC in response to Question #5 and provides the following additional responses.  ERCOT remains concerned that the SRC’s points on the standards grading methodology expressed in the first round of comments have not been addressed or acknowledged.  Of most concern from the SRC’s previous comments is that the grading methodology is flawed.

There are existing problems with standards language that are not accounted for in the standard grading criteria.  Three of the questions most impactful to this effort are not even graded.  The three: 1) Supports a Reliability objective, 2) Meets Paragraph 81 criteria, and 3) Appropriate as a guide rather than a standard, are crucial to determine the usefulness of a standard, but are not factored into the final grade.  Answers to any these questions would automatically necessitate either a revision, retirement, or conversion to a guide, and ERCOT questions why these important criteria are not factored into the final grade for a standard.

ERCOT asserts that several of the criteria don’t actually indicate standard quality.   For instance, the criteria “Is it complete and self-contained,” measures two very different criteria.  While completeness of a standard is a reasonable measure of quality, “self-containment” is not necessarily an indicator of a quality or clarity problem.  There are many NERC standards that reference other standards; for instance, the TPL standards rely on data collected through MOD standards, or, RC/BA requirements mirrored in TOP requirements require a response/reaction from each functional entity to work together effectively to impact grid operations.  ERCOT questions what other option is available for cross-referencing standards if the subject matter is related.  What other option exists? Therefore, ERCOT questions the usefulness of this metric, as well as its questionable coupling with the “completeness” metric.

To grade the quality of the standards, ERCOT suggests using an alternative method, and one that can be performed via review of a simple list of questions from the current criteria.  Questions such as “Is the language clear and does not contain ambiguous or outdated terms?” or “Are the correct functional entities identified?” should be included in this list. If the answer to any of these questions is “no,” this should trigger a revision of standard, rather than contributing an unidentified weight of importance summed into scores that are further aggregated, obscuring the actual response value in the first place.

Elizabeth Axson, On Behalf of: Elizabeth Axson, , Segments 2

- 0 - 0

Southern Company, Segment(s) 1, 6, 3, 5, 9/1/2016

- 0 - 0

Other Answers

No comment.

Daniel Grinkevich, On Behalf of: Daniel Grinkevich, , Segments 1, 3, 5, 6

- 0 - 0

To date, the scores don't seem to align with the comments that are imbedded with the grades. For example, several of the requirements have comments such as "Would be acceptable as a guideline". If reliability could be achieved as a guideline, the Standard should not receive a high score. These type of comments within the grading seem counterintuitive to scoring a Standard high.

Ashley Stringer, On Behalf of: Oklahoma Municipal Power Authority, , Segments 1, 4

- 0 - 0

None

Aaron Cavanaugh, On Behalf of: Bonneville Power Administration, WECC, Segments 1, 3, 5, 6

- 0 - 0

Robert Ganley, On Behalf of: Robert Ganley, , Segments 1

- 0 - 0

The TPL-001-4 standard presently requires the assessment of sensitivity cases for the extreme event contingencies in addition to the assessment baseline cases. Since extreme event analysis results are used for information purposes, rather than requiring corrective action plans, this significant amount of additional work is a resource burden with inadequate benefit and should be eliminated.

Lauren Price, On Behalf of: Lauren Price, , Segments 1

- 0 - 0

sean erickson, On Behalf of: Western Area Power Administration, , Segments 1, 6

- 0 - 0

The following organizations concur with the comments listed below:

PJM, ISO-NE, MISO, IESO, CAISO, NYISO, SPP and ERCOT

General Comments:

The ISO/RTO Council Standard Review Committee (SRC) appreciates the work of the PRSRT team. While the enhanced review and scoring is fine in concept, one fundamental consideration is that the review should only be performed on standards that either have been directed to be changed or that are coming up on their 10 year review.  Additionally, for those that are on their 10 year review, the Industry should have a vote on whether the standard is acceptable as-is.  There should be a vote prior to opening a new project. Give the Industry the opportunity to vote on whether they want to keep the given standard “as-is”. If there are minor problems, such as paragraph 81 requirements, they can be handled in compliance.

The SRC believes that Reliability Standards have come a long way since their inception ten years ago and FERC’s Order 693.  NERC’s approaches to implementing “Steady-State” Standards and the Risk-Based Compliance Monitoring have brought much needed clarity and efficiencies in the understanding and enforcement of standards.  Certainly, as new threats to the reliability & security of the grid arise, and new findings are made from the analysis of system disruptions and events, NERC must continually be vigilant to determine what means are best suited to mitigate these threats with standards being one of the many tools to be used.

In that light, our primary concern with the PRSRT is that NERC too heavily emphasizes creating and revising Reliability Standards. NERC should guard against expending its resources as well as those of stakeholders and registered entities from counteracting the benefits of the Steady State Standards and Risk-Based Compliance monitoring initiatives. The development, implementation and enforcement of standards have cost implications that reverberate through the NERC organization, registered entities, and ultimately customers.   Though the Steady State Standards effort has ended, the revisions are still relatively new and have little data and documentation to show how effective they are in affecting reliability.  NERC should not revise the approved standards unless there are demonstrable deficiencies.  NERC must realign its efforts to first assess how effective the standards are.  The Enhanced Periodic Review process should include steps to assess true impacts on reliability through acquiring relative grid data.

The SRC is not saying that the current set of NERC standards is perfect and no further periodic reviews or changes are needed.  The efforts NERC and the industry have taken in recent years appear to have achieved major improvements in the level of reliability and compliance. Directing industry to expend additional resources on new or modified standards should be done primarily where NERC is able to identify how the change significantly improves reliability or reduces unnecessary compliance burden.  NERC should work with RISC or NERC’s Standing Committees to develop a strategic approach to identify where significant gaps in Reliability Standards may exist and how best to address those gaps.  Conducting this type of strategic analysis will help NERC and Stakeholders understand what type of Standard Development work, compliance monitoring approach or other awareness enhancing activities are appropriate.

Detailed Comments:

  1. NERC’s PRSRT idea is good, but the SRC believes the Standard grading tool must be improved with some mechanism to account for the importance of certain questions. The PRSRT assumes each of the factors has exactly equal weight, but this is not the case. Some questions are extremely important. For example, if the standard does not identify the correct functional entities (Col. H of the grading matrix), cannot be practically implemented (Col. T), or is not cost-effective (Col. V), then any one of these factors should presumably justify immediately fixing (or retiring) the standard. These factors can’t be “outweighed” by good scores, or even perfect scores, on other factors. A reasonable approach would be to have the PRSRT consider each of the factors as part of a holistic review, recognizing that some of the factors may be significant enough by themselves to justify re-opening the standard.

  2. The SRC questions why there is a need to build score consensus? If the OC, PC, RE, and NERC personnel complete their assessments in good faith, then the score should remain exactly as it is without deference to consensus building. Consensus in the worst case simply means that compromise is artificially imposed when compromise is not always necessary. For example, there may be logical and defensible reasons for severe differences between OC and PC scores and these scores should remain as-is.

  3. Simply tallying “yes’s” and “no’s” doesn’t yield a true or effective result.  Taken individually, some of the criteria, if graded poorly, negate the entire quality of the standard.  Yet some questions are pivotal to judge the effectiveness of a standard.  If all other criteria are judged positively, the grading outcome does not accurately reflect the deficiencies of the standard.

  4. The criteria in the PRSRT ranking tool mentions “risk” only twice, which seems inadequate given NERC’s risk-based compliance efforts. Risk to the BES should be measured, in line with the risk-based compliance oversight.  These criteria, perhaps more than any other, should be used to help prioritize and improve standards.

  5. The ranking/grading by respondents is a flawed methodology. The SRC requests clarity on the reasoning for a quality rating of 5 versus a quality rating of 10. What score determines whether a Standard will or won’t be updated?

  6. The quality criteria Q11, “can it be practically implemented?” may receive a number of different answers from respondents depending on the capability and definition of “practical” of each respondent, and isn’t measureable. How did each individual make their ranking determination? Neither the published content or quality criteria considered topics like:

    1. Violation statistics

    2. Risk-based compliance concepts (focus on High Risk)

    3. Consistency with other Reliability Standards

    4. Changes in Technology and System Conditions

    5. Risk and/or Events on the BES system

  7. It is not completely clear to the SRC what the PRSRT is going to do with the ranking/grading and how the information will be used, especially in light of the existing information contained in the “Periodic Review Template” (See: http://www.nerc.com/pa/Stand/Standards%20Development%20Plan%20Library/Enhanced_Periodic_Review_Template_090214.pdf ) updated in September 2014.

  8. The Frequently Asked Questions (FAQ) document (See: http://www.nerc.com/pa/Stand/2017%20Periodic%20Review%20Standing%20Review%20Team%20%20Standar/2017_PRSRT_FAQ_document_06192017.pdf) notes in Q1/A1 that “The finalized grading will be appended to the Reliability Standards Development Plan (RSDP), which has been endorsed by the SC.”, but what this means and how it will help determine whether a Standard will be updated or not is still in question.

  9. The SRC questions the value to the PRSRT effort in general. How does it differ or provide more information to the existing Periodic Review Template, which was designed to give a “red, yellow, or green” grading for a particular Standard. Will the new EPRSRT tool replace the “Periodic Review Template”? If not, why not update the template first, with industry review, comment, and approval, and then apply the new tool to the periodic review of Standards?

  10. The risk triage process performed by Enforcement staff should be transparent and objective (e.g. serious violations are those that are related to “high impact” requirements known to cause system events or where there was an observed material impact on reliability). NERC should use event data and observations from violations (where there was an observed impact on reliability) to identify the true “high impact” requirements that would be the focus of compliance monitoring, and the development/sharing of internal controls. If there is not a clear trend threatening a BES Benchmark, a Standard is probably not the correct solution.

IRC-SRC, Segment(s) 2, 8/2/2017

- 0 - 0

PPL NERC Registered Affiliates, Segment(s) 3, 1, 5, 6, 2/9/2017

- 0 - 0

We agree with APPA's comments:

Since it appears that the new Paragraph 81 effort would include consideration of the Periodic Reviews (PR) public power believes that NERC should establish how the efforts will work together to ensure efficiency.  Stakeholders will not be well served providing comments on duplicative efforts addressing similar issues.

For example, initial new Paragraph 81 discussions have included retirement of standards or specific standard requirements.  If the Periodic Reviews" are intended to include "retirement" the PR process needs to offer more diversity in the scores. The draft 2017 evaluation and grades show generally high scores for all reviewed standards. Often very high, close to perfect scores have been awarded; for example, NERC gave a perfect Quality score to all 47 requirements reviewed. Consequently, the results do little to distinguish a standard or requirement, in a way that suggests there will be robust discussion on the standard being reviewed.  

The Independent Experts Review Process (IREP) was a more pointed review, suggesting paths forward for specific standard families or requirements, including retirement.  Public power recognizes that the PR grading is a data point in a more comprehensive review of standards.  However, on their face, the current matrix results do not offer much dispersion in the scores that would prompt discussion, let alone directed discussion that would necessitate changes to, or retirement of, existing standards.  

FMPA, Segment(s) , 8/2/2017

- 0 - 0

Exelon agrees with the comments provided by the RTO / ISO Council and urge NERC to consider their recommendations for future Grading or Periodic Review initiatives.

Daniel Gacek, On Behalf of: Exelon, , Segments 1, 3, 5, 6

- 0 - 0

Illinois Municipal Electric Agency agrees with comments submitted by American Public Power Association.  

Mary Ann Todd, On Behalf of: Illinois Municipal Electric Agency, , Segments 4

- 0 - 0

The SRT should review the category numbering under Quality Score, it notes 0-12 on the RE tab, and there are some listed with a grade of 13.

In addition, if a Standard is under a SAR, undergoing a currently active drafting team, or with an effective date in the current year, these should not be included in the review at these early stages.

Furthermore, the rationale/guidance documents should be considered in the grading process. The same evaluation given to the standards should be conducted on the rationale/guidance documentation to provide consistency for the protection of the reliability of the grid.

 

SPP Standards Review Group, Segment(s) , 8/2/2017

- 0 - 0

Duke Energy , Segment(s) 1, 5, 6, 4/10/2014

- 0 - 0

  1. We thank the SRT for providing outreach opportunities between various industry sectors of the NERC Technical Committees to grade the identified Reliability Standards. However, the process still consolidates this input into two distinct sample sets equally assessed with the NERC and Regional Entity.  Moreover, the consolidation removes any statistical outliers and modifies initial individual committee member assessments with after-the-fact collective committee discussion.  We believe four data sets are statistically insufficient for data collection, as many statistical textbooks identify that a sampling set of at least 30 is necessary.  At a minimum, the process should not consolidate any input provided.
  2. We question how a process can identify a content and quality score of 100%, yet disregard responses to the three general questions regarding Reliability Objectives, Paragraph 81 criteria, and appropriateness for guide development.  If a requirement is identified as meeting the Paragraph 81 criteria, then a project should be assigned to retire that requirement regardless of other grading identified.
  3. We believe some of the questions have identical meanings that unfairly weigh those responses with other questions.  For instance, how different is the content question “identifying who does what and when” from the question regarding the identification of the correct functional entity?  Likewise, the quality question asking if the requirement is “complete and self-contained” is nearly identical to the question asking if the requirement is “stand-alone” or should it be consolidated with other standards.  What reference materials are available that provide background and the expectations associated with answering the content and quality questions, and has these materials been provided to the SRT?
  4. There are no questions available to identify Violation Risk Factor misalignments or incomplete Violation Severity Limits.  For example, we believe requirement R4 of EOP-011-1 could inadvertently place a significant financial burden on a TOP who is required to resubmit its Operating Plans back to its RC, particularly if the RC has identified an unachievable time period (i.e. same day).   Under such conditions, the TOP would violate the requirement based on its High Violation Risk Factor and High Violation Severity Level.  We feel the failure to update an Operating Plan is administrative in nature, and should instead be classified as a low Violation Risk Factor.
  5. We thank you for this opportunity to comment

 

ACES Standards Collaborators, Segment(s) 1, 3, 5, 4, 8/2/2017

- 0 - 0

The questions for quality and content as indicated on the Grading Workbook are ambiguous in many cases and should be clarified in future versions of Standard Grading Activities.

RSC, Segment(s) 1, 0, 2, 4, 5, 6, 7, 3, 7/18/2017

- 0 - 0

OPG is in agreement with revision of future Standard Grading questions quality and content, to remove the generality aspect.

David Ramkalawan, On Behalf of: David Ramkalawan, , Segments 5

- 0 - 0

Rick Applegate, On Behalf of: Rick Applegate, , Segments 1, 3, 4, 5, 6

- 0 - 0

Since it appears that the new Paragraph 81 effort would include consideration of the Periodic Reviews (PR) public power believes that NERC should establish how the efforts will work together to ensure efficiency.  Stakeholders will not be well served providing comments on duplicative efforts addressing similar issues.

For example, initial new Paragraph 81 discussions have included retirement of standards or specific standard requirements.  If the Periodic Reviews" are intended to include "retirement" the PR process needs to offer more diversity in the scores. The draft 2017 evaluation and grades show generally high scores for all reviewed standards. Often very high, close to perfect scores have been awarded; for example, NERC gave a perfect Quality score to all 47 requirements reviewed. Consequently, the results do little to distinguish a standard or requirement, in a way that suggests there will be robust discussion on the standard being reviewed.  

The Independent Experts Review Process (IERP) was a more pointed review, suggesting paths forward for specific standard families or requirements, including retirement.  Public power recognizes that the PR grading is a data point in a more comprehensive review of standards.  However, on their face, the current matrix results do not offer much dispersion in the scores that would prompt discussion, let alone directed discussion that would necessitate changes to, or retirement of, existing standards. 

Jack Cashin, On Behalf of: American Public Power Association, , Segments 3, 4

- 0 - 0

Sergio Banuelos, On Behalf of: Tri-State G and T Association, Inc., MRO, WECC, Segments 1, 3, 5

- 0 - 0