Special Notice / Hearing: None__
Vote Required: Majority
To: Honorable Board of Supervisors
From: John T. Keene, Chief Probation Officer
Subject: Waiver of the Request for Proposal Process and Acceptance of an Agreement with Cognisen
RECOMMENDATION:
title
Adopt a resolution authorizing:
A) A waiver of the Request for Proposal process; and
B) The execution of an agreement with Cognisen to provide DocAssist and FieldAssist artificial intelligence tools for the term of March 1, 2025 through February 28, 2030 in an amount not to exceed $2,654,568.91; and
C) The Chief Probation Officer or his designee to execute contract amendments and minor modifications not to exceed $25,000 and/or modify the contract term and/or services.
body
BACKGROUND:
In January 2024, the Board of Supervisors adopted a resolution affirming the importance of maintaining County jobs during advancements in artificial intelligence (AI). The Board of Supervisors acknowledged the potential benefits of incorporating AI technologies in the County’s operations for increased efficiency and service improvement. The Probation Department’s agreement with Cognisen demonstrates a clear commitment to this resolution by ensuring that any AI implementation will focus on assisting staff, not replacing them.
In July 2024, the Probation Department entered into a pilot agreement with Cognisen for the contractor to demonstrate the capabilities of their PEARL AI. This collaboration aimed to evaluate the potential of Cognisen's PEARL technology, specifically DocAssist and FieldAssist, to enhance the efficiency and effectiveness of Deputy Probation Officers (DPOs). DocAssist was designed to streamline court report drafting, while FieldAssist focused on optimizing field supervision practices. Throughout the pilot period, Cognisen worked closely with the Probation Department to customize templates, workflows, and system configurations, ensuring that the AI implementation aligned with departmental standards, addressed key operational challenges, and ultimately improved service delivery.
DISCUSSION:
Upon execution of this five-year agreement, Cognisen will continue to refine the DocAssist tool and begin to build out the FieldAssist tool uniquely for the Probation Department.
Court report writing is a critical function of the Probation Department, requiring DPOs to meticulously document a defendant’s criminal and social history. DocAssist aims to streamline this process by automating repetitive tasks, improving efficiency, enhancing accuracy and consistency. Importantly, the AI tool will be used as an assistive technology, with DPOs maintaining full oversight and responsibility for the final content of all reports.
DocAssist’s features support sole-source procurement. It is the only AI-powered report generation tool specifically designed for the community supervision field that integrates with any case management system and significantly reduces report generation time while ensuring compliance with department standards and guidelines. DocAssist and FieldAssist are fully compliant with Criminal Justice Information Systems (CJIS) and meets all the stringent security and privacy requirements necessary for handling sensitive criminal justice information.
The forthcoming FieldAssist tool will transform field operations for supervision officers by streamlining client prioritization, optimizing routes for field operations, and automating routine case documentation. By providing real-time case insights and automating routine tasks, FieldAssist will empower DPOs with the necessary tools for efficient, data-driven supervision in the field. This innovative solution not only simplifies client compliance tracking, but it also enhances officer safety by providing instant access to critical information, such as warrants and safety notes, prior to each client interaction. Ultimately, FieldAssist enables DPOs to spend more time on impactful, face-to-face interactions, improving service delivery and client accountability.
The resolution and agreement have been reviewed and approved as to form by County Attorney.
The following responses address the Board of Supervisors’ AI software questionnaire:
Risk Management
1. Does the vendor carry adequate Product Liability and General Liability Insurance? Yes. Cognisen carries insurance that either meets or exceeds the county’s liability insurance requirements and has provided a certificate of insurance with San Mateo County listed as additional insured as per county requirement.
2. Does the vendor indemnify the County for all errors and omissions created by its software? In section 13 of the contract, Cognisen agrees to Indemnify San Mateo County. While Cognisen takes every precaution to ensure the accuracy and reliability of its software, it is ultimately a report creation tool that is specifically designed to ensure that DPOs maintain complete control and decision-making responsibility over all AI-assisted content. Cognisen generates AI-assisted draft content that acts as a starting point for DPOs, requiring their review, edits, and final approval before submission. DPOs’ professional judgment remains central to the process, as they are responsible for tailoring the content to reflect their expertise and insights. Additionally, Cognisen includes clear workflows to ensure that all final content is attributed to the DPO, reinforcing their accountability and authorship. By design, no document is finalized without direct DPO input and approval, ensuring the AI serves as a supportive tool rather than a replacement for human oversight.
3. Does the contract carry a provision that the County will be immediately notified of any material complaint the vendor receives from any other customer related to the performance of the software? The contract does not contain such a provision. However, Cognisen is deeply committed to maintaining a strong, transparent, and trusted relationship with San Mateo County.
The probation community in California is a highly collaborative network, with integral associations such as the Chief Probation Officers of California (CPOC) and its subgroups, including the Probation Business Managers Association (PBMA), the Probation IT Managers Association (PITMA), and the California Association of Probation Service Administrators (CAPSA). Cognisen is actively involved in these associations, where experiences with software vendors and their applications are openly discussed among counties. San Mateo County’s active participation in these organizations ensures that county leaders stay informed about vendors, best practices, and any challenges faced by peer agencies. This includes Cognisen and its software solutions and its customer base routinely share their insights and experiences.
4. Is there a termination provision in the contract should the County determine the AI software implemented is no longer state of the art? Yes. The contract includes a provision that allows the County to terminate the agreement with written notice of non-renewal of at least 120 days prior to the expiration of any current, 12-month term.
5. What other jurisdictions have adopted the use of this software? If less than 10 jurisdictions are currently using the AI software, when did they start using the AI software? San Joaquin County (Pilot County - Live October 2024), Stanislaus County (Pilot County - Live January 2025), and Lake County (contract signed December 2024)
Employee Impact
6. What human action will the AI software replace? Cognisen’s software does not replace any human action; rather, it serves as an intelligent assistant to probation officers, enhancing their efficiency while ensuring they retain full control and decision-making authority. The software generates AI-assisted draft content in specific areas, acting as a starting point for officers who are responsible for reviewing, editing, and finalizing all documents before submission. Officers’ professional judgment remains central to the process, ensuring that reports and other documentation accurately reflect their expertise, insights, and case knowledge.
Additionally, Cognisen’s software includes structured workflows that reinforce officer accountability by requiring explicit review and approval before any document is finalized. This ensures that every report remains under the officer’s authorship and responsibility. By design, the software functions as a supportive tool to reduce administrative burdens-not as a replacement for human oversight or expertise.
7. How many new hire employees is it estimated the AI software eliminates the need for, presently? Cognisen’s software does not eliminate the need for new hires, but it helps mitigate the challenges caused by persistent staffing shortages. Like many probation departments across California, San Mateo County faces a high turnover rate due to retirements, career transitions, and difficulties in recruiting and retaining qualified personnel. These staffing challenges create a growing workload that makes it harder for officers to effectively supervise individuals on probation.
Cognisen’s software alleviates this burden by reducing the time required for documentation, streamlining workflows, and improving report consistency-allowing officers to spend more time on high-value tasks like supervision, intervention, and rehabilitation efforts. By assisting with drafting reports and ensuring a consistent level of quality regardless of an officer’s experience level, Cognisen helps maintain operational effectiveness even when staffing levels fluctuate.
Rather than replacing the need for personnel, the software enables departments to function more effectively with their current workforce, ensuring that probation officers can focus on their core mission: providing better supervision, accountability, and rehabilitation-ultimately enhancing public safety for the residents of San Mateo County.
8. Does the AI software eliminate the position of any current employees? No, Cognisen’s software does not eliminate any current positions. While it allows departments to complete tasks more efficiently, the outcome is not job loss but rather a strategic reallocation of staff to higher-priority responsibilities. For example, within the Adult Investigations unit, officers currently spend a significant portion of their time drafting Pre-sentence Investigation Reports (PSRs). With Cognisen’s software assisting in this process, the time spent creating these reports can be reduced by 50% to 80%, which will allow leadership to redirect staff efforts toward higher-impact activities such as supervision, case management, and intervention strategies.
Validation / Quality Assurance
9. What type of Artificial Intelligence is being used by the software? Cognisen’s software utilizes a Generative, Large Language Model (LLM)-specifically, Anthropic’s Claude 3.5 Sonnet, hosted in AWS GovCloud’s Bedrock. The model assists in generating structured content within report workflows based on user inputs and predefined templates.
10. Has the AI model undergone independent third-party validation? Yes, Anthropic has conducted independent validation, testing, and auditing of the Claude 3.5 Sonnet model, ensuring it meets industry standards for accuracy, reliability, and bias mitigation. Additionally, AWS provides security and compliance assurances for AI models deployed within Bedrock.
11. How has the AI model been validated?
Anthropic's Internal Evaluation: Anthropic rigorously evaluates Claude models through various methodologies, including adversarial testing, reinforcement learning from human feedback (RLHF), and benchmarking against standardized NLP datasets.
Bias and Fairness Testing: Claude 3.5 has been evaluated for bias using diverse datasets to mitigate harmful outputs and ensure responsible AI behavior.
Internal Testing and Validation by Cognisen’s Team: (a) Conduct regular accuracy checks against known report formats and legal frameworks, (b) Cognisen’s software includes user-driven validation-users can review, edit, and provide feedback on AI-generated content, which informs our continuous improvement processes, and (c) Reports are randomly sampled and audited for accuracy, completeness, and fairness.
12 . What was the AI model’s validation performance? Claude 3.5 Sonnet
has demonstrated high accuracy and consistency in content generation, scoring competitively against leading benchmark datasets in reasoning, factuality, and coherence.
13. Does the contract require the County be notified of the software using new AI models implemented after procurement and their validation performance? No, the contract does not explicitly require prior notification of AI model updates. However, Cognisen maintains transparency regarding AI usage and can provide periodic updates upon request if new AI models are integrated into the system.
14. How will the County department formally monitor the accuracy and performance of the AI software both for errors and subjective bias?
Cognisen’s software provides multiple mechanisms for ongoing accuracy and bias monitoring:
a. User Review & Feedback Loop - Users can flag AI-generated content within reports, triggering internal audits for quality assurance.
b. Audit Logging & Transparency - AI-generated content is timestamped and logged, allowing administrators to review and assess model outputs over time.
c. Automated Performance Monitoring - The system tracks AI output against historical report data to identify anomalies, inconsistencies, or deviations.
d. Periodic Quality Audits - The department will conduct randomized sampling and validation of reports to assess the model’s accuracy and fairness.
e. Customizable AI Tuning - Cognisen can adjust AI-generated outputs based on department feedback to better align with agency requirements.
Data
15. Does the AI model train on data provided by the County? No. Cognisen will never use San Mateo County data to train underlying AI models. Cognisen uses a combination of carefully compiled contextual data and sophisticated prompt engineering to achieve reliable draft content to assist users in performing very specific tasks assigned to them by the Probation Department. Once the model has generated the requested content, all contextual data is flushed from memory.
In the event that additional model fine tuning becomes necessary, it will be accomplished using synthetic data that does not contain any personally identifiable information (PII) or Criminal Justice Information (CJI).
16. Will the County be transferring any County data to the Vendor to be kept offsite or to Cloud? Yes. Cognisen’s software runs in AWS GovCloud (US) as a SaaS software application. This ensures the highest level of security and compliance for data transmission and storage. All data is encrypted at rest and in transit and meets Criminal Justice Information Systems (CJIS) standards as set forth by the FBI.
17. If so, how is the privacy of County residents being protected? Cognisen adheres to CJIS standards and leveraging AWS GovCloud (US) to ensure data remains secure and within the United States. All data is encrypted in transit and at rest using FIPS 140-2 validated encryption, and access is restricted to U.S. persons with proper authorization. Cognisen’s environment is continuously monitored for security threats, with role-based access controls, multi-factor authentication, and full audit logging to ensure compliance and accountability. County data is never stored or processed outside of AWS GovCloud, providing a secure, CJIS-compliant environment that protects resident privacy.
18. When the contract ends, will all County data be returned to the County, and all copies of the data in possession of the vendor, destroyed? Yes, the contract specifically addresses this issue in section 15.4(b).
19. Will any County data be stored in a foreign jurisdiction? No. Cognisen utilizes AWS GovCloud (US) to ensure the highest level of security and compliance. AWS GovCloud (US) data centers are located exclusively within the United States and are designed to meet the stringent regulatory and compliance requirements of federal, state, and local government agencies. This ensures that all County data remains on U.S. soil and is protected by robust security controls, including CJIS compliance, and other critical government security standards. Additionally, AWS GovCloud (US) restricts access to U.S. persons, further reinforcing data sovereignty and security.
PERFORMANCE MEASURE:
|
Performance Measures |
FY 2024-25 Target |
FY 2025-26 Target |
FY 2026-27 Target |
FY 2027-28 Target |
FY 2028-29 Target |
|
Percent of Deputy Probation Officers reporting a reduction in court report generation time |
95% |
95% |
95% |
95% |
95% |
|
Percent of Deputy Probation Officers reporting satisfaction with FieldAssist’s ease of use, functionality, and impact on daily work |
95% |
95% |
95% |
95% |
95% |
EQUITY IMPACT:
The implementation of Cognisen's AI technology has the potential to promote equity by enhancing resource allocation and streamlining operations. If effectively implemented and monitored, the DocAssist and FieldAssist tools could optimize resource allocation, potentially ensuring that limited resources are directed towards individuals with the greatest needs. Automating routine tasks may free up staff time for more impactful work, such as providing individualized support to clients and building stronger community relationships.
FISCAL IMPACT:
The term of the agreement is March 1, 2025 through February 28, 2030, in an amount not to exceed $2,654,568.91; and authorizes the Chief Probation Officer or his designee to execute contract amendments and minor modifications not to exceed $25,000. Funds for this agreement have been included in the FY 2024-25 Adopted Budget and will be included in future budgets for the term of the contract.