Dear Leif, ARB, Lynzie, Regarding the changes being implemented in SAH v4.0, I’d like to make a few observations on process and potential consequences of the changes made, for your consideration. As I understand Kantara’s organization: a) The IAWG owns and sets the assessment criteria – it effectively owns the ‘scheme’; b) Assessors review CSPs’ evidence to determine whether (or not) conformity is found against the assessment criteria; c) The ARB exists to ensure that assessments executed against the assessment criteria are performed correctly and at a consistent level across all Assessors / Classes of Approval; d) The ARB is entitled to ask Assessors to present their findings in a particular fashion to help the ARB in its assigned responsibilities and to foster consistency in the application of the assessment criteria; e) This reflects the principals of IS17065. On the basis of the above, I see some problems arising from the SAH revisions: 1) The ARB (rather than the IAWG) appears to be setting rules for the creation by Assessors of assessment criteria (the ‘tests’, per the SAH). This fractures the IAWG / ARB division of responsibilities; 2) It leads to different Assessors potentially (likely, I would say) determining different tests for the same or a very similar subject, such that: a. for any test subject supporting CSPs, there will be no consistency; b. not having a single ‘spec’ for testing any specific supporting service will lead to inefficiency; 3) Providers of the potential test subjects are unlikely to want to play the game so any testing will be of a ‘black box’ nature; 4) Assessors now start to rely on tests (read ‘assessment criteria’) and implicitly on expected test results which they have themselves drafted and which therefore are perfectly good, couldn’t possibly fail [insert preferred sarcasm emoticon here]. This seems to be a potential transgression of the assessors’ need to not be assessing their own work; 5) Such tests can only reasonably be applied within the scope of the requirement of any applicable assessment criteria – I really don’t think that assessors should go about creating ‘new’ tests/criteria or ‘layering’ existing criteria upon those test subject providers; 6) There are cost implications for assessors to define the tests, to conduct the tests and for CSPs to support the tests. Noting the ARB’s short-term expectation of when the changes should become effective, assessments which have already been contracted will almost certainly need to be renegotiated. This will not be welcomed by CSPs, and I daresay nor by assessors; I think it unfortunate that there was no consultation of these changes with assessors or the IAWG at large. Had there been, perhaps some of these points could have been addressed and potential problems mitigated before a definitive document was produced. I note also that the email of June 21st was sent only to assessors. Furthermore, IF these sorts of ideas are to be established then CSPs need fair notice from KI, NOT from the Assessors (let KI be the purveyor of this news ;-). As far as I am aware, CSPs have yet to be advised of these new expectations, which restricts everyone’s ability to address/react to the changes. May I suggest that a more stable solution to the need identified by the ARB (and I don’t disagree with the principle point being raised) would be the development of additional criteria which can be uniformly understood and applied, and that such criteria be developed by the ARB? The matter of when such criteria should come into force should also be addressed where the work associated with assessments is being increased, rather than only ‘tweaking’ criteria interpretations or their applicability. Thank you for considering this ten cents-worth. Richard G. WILSHER CEO & Founder, Zygma Inc. www.Zygma.biz +1 714 797 9942 From: Lynzie Adams [mailto:lynzie@kantarainitiative.org] Sent: Wednesday, June 21, 2023 19:19 To: Jimmy Jung; Richard G. WILSHER (@Zygma); Scott Perry; Ray Kimble; beltus.ikechi@kuma.pro Subject: Letter from ARB and Updated Kantara Service Approval Handbook Hello Assessors, The ARB recently approved an update to the Service Approval Handbook https://kantarainitiative.org/download/7590/ . I uploaded it to Kantara's website today and it will be the required version on October 1. Please take an opportunity to review the updates, which include: This revision incorporates these material changes: * removed “-fulfilled by a pre-approved service” as an Assessor’s SoC finding and corresponding guidance (§3.4.1); * introduced “Registered Applicant Fee” (§5.1, §5.2.3); * added special provision for Non-Kantara Approved Services (§6.1.1.3) (see attached letter) * increased duration that a service must be operating before a Fully Operational Service Assessment shall be completed (§6.1.2.1; §6.1.2.3); (from 60 to 90 days) * redefined a Fully Operational Assessment once RTO status lapses (§6.1.2.3); (it's now a triennial assessment after the RTO lapses) * requirements for Assessor on-site visits (§6.1.3; §8.1.3); and (now a SHOULD with justification if site visit does not occur) * updated the due date of renewal applications for Triennial or ACR assessments (§8.2). (due at the beginning of the month, rather than the end) Additionally, the following non-material changes have been embodied: * changed “Registered Service” status to “Registered Applicant” (§5); * relaxed target completion date for ARB reviews (§6.2.1.1); and * revisions throughout to increase clarity of the process. I'm also happy to answer any questions about what the updates include. I will be sharing the new version and updates with my CSP contacts as well. Further, please see the attached letter from the ARB about the use of non-Kantara Approved services, which is also referenced in the SAH update. Additionally, the ARB's next task is to update the Assessor Accreditation Handbook. I'll reach out when those updates are complete. Lynzie Adams Assurance Program Manager https://docs.google.com/uc?export=download&id=1f_FyEBepOKYcAfkyIsaSFzguA7Tg8XgN&revid=0B7AAtl15W-dSdms4Mjk0bVhuUmNDOHZGTjd6Y3dMcDFrRDhvPQ
