How Can Machine Learning Models for SIs Be Governed to Prevent Bias?

Concept

The imperative to govern machine learning models within Systematic Internalisers (SIs) is a direct function of the system’s core purpose to provide reliable, firm liquidity. Bias in these models represents a systemic threat to this function. When an SI’s pricing or routing algorithms develop biases, they are not merely statistical anomalies, they are corruptions of the SI’s commitment to provide fair and consistent pricing.

This can manifest as subtle, yet persistent, deviations in quote quality for certain client segments or under specific market conditions, directly impacting execution quality and eroding trust. The governance of these models is therefore an exercise in preserving the integrity of the SI’s market-facing operations.

At its core, bias in an SI’s machine learning model is a deviation from an expected outcome that can be traced to the data used to train the model or the design of the model itself. These are not abstract statistical concepts; they have tangible consequences for market participants. For example, a model trained on historical data that includes a period of unusual market stress might learn to penalize certain types of orders, even when market conditions have normalized.

This can lead to a situation where the SI systematically provides less favorable quotes to a specific group of clients, not out of malice, but because the model has been trained to associate their trading patterns with higher risk. This is a direct violation of the SI’s obligation to provide fair and non-discriminatory access to its liquidity.

A primary objective of a machine learning model framework is the reduction of societal bias to the lowest possible level.

The Anatomy of Algorithmic Bias in Systematic Internalisers

To effectively govern machine learning models, it is essential to understand the different forms that bias can take. In the context of SIs, bias can be broadly categorized into two types data-driven bias and model-driven bias.

Data-Driven Bias

Data-driven bias arises from the data used to train the model. This can include:

• Historical Bias This occurs when the data used to train a model reflects existing biases in the real world. For example, if a model is trained on historical trade data that shows a particular asset class to be more volatile, it may learn to systematically overprice that asset, even if its current volatility is low.
• Selection Bias This type of bias occurs when the data used to train a model is not representative of the population it will be used to make decisions about. For example, if an SI’s model is trained primarily on data from large, institutional clients, it may not perform well when providing quotes to smaller, retail-focused brokers.
• Measurement Bias This arises from inaccuracies in the data itself. For example, if the data used to train a model contains errors in trade timestamps or prices, the model may learn to make inaccurate predictions.

Model-Driven Bias

Model-driven bias, on the other hand, is a result of the model’s design or the way it is used. This can include:

• Algorithmic Bias This occurs when the model’s algorithm itself is flawed. For example, a model that is too complex may overfit to the training data, meaning it will perform well on the data it was trained on but poorly on new data.
• Evaluation Bias This arises when the metrics used to evaluate a model’s performance are not appropriate for the task at hand. For example, if a model is evaluated solely on its accuracy, it may not be clear whether it is making fair and unbiased decisions.

Strategy

A strategic framework for governing machine learning models in Systematic Internalisers must be built on a foundation of transparency, accountability, and continuous monitoring. The goal is to create a system where bias can be identified, measured, and mitigated at every stage of the model lifecycle, from data collection and model development to deployment and ongoing performance monitoring. This requires a multi-faceted approach that combines technical solutions with robust governance processes and a culture of ethical AI.

The core of this strategy is the development of a comprehensive model risk management framework that is specifically tailored to the unique challenges of machine learning. This framework should be based on the following key principles:

A commitment to responsible AI practices builds a positive reputation, leading to increased customer trust and loyalty.

A Multi-Layered Governance Framework

A robust governance framework for ML models in SIs should consist of multiple layers of defense against bias.

First Line of Defense the Model Development Team

The first line of defense is the team responsible for developing and implementing the model. This team has a critical role to play in ensuring that the model is fair and unbiased. Key responsibilities include:

• Data Governance Ensuring that the data used to train the model is accurate, complete, and representative of the population it will be used to make decisions about.
• Fairness by Design Incorporating fairness considerations into the model design process from the very beginning. This includes selecting appropriate fairness metrics, using bias mitigation techniques, and documenting all decisions made during the model development process.
• Explainability Building models that are transparent and explainable, so that it is possible to understand how they make decisions. This is essential for identifying and mitigating bias, as well as for complying with regulatory requirements.

Second Line of Defense the Model Validation Team

The second line of defense is an independent model validation team. This team is responsible for assessing the model’s performance and ensuring that it is fit for purpose. Key responsibilities include:

• Independent Review Conducting a thorough and independent review of the model, including its data, methodology, and performance.
• Bias Testing Using a variety of techniques to test the model for bias, including statistical tests, scenario analysis, and fairness audits.
• Model Documentation Ensuring that the model is well-documented, so that it is possible to understand how it works and how it was validated.

Third Line of Defense Internal Audit

The third line of defense is the internal audit function. This team provides an independent assessment of the effectiveness of the model risk management framework. Key responsibilities include:

• Auditing the Framework Auditing the model risk management framework to ensure that it is well-designed and operating effectively.
• Reporting to the Board Reporting the results of their audits to the board of directors and senior management.

What Are the Key Regulatory Considerations?

Financial institutions must navigate a complex web of regulations when implementing machine learning models. Key regulations include the Equal Credit Opportunity Act (ECOA) and the Dodd-Frank Act’s prohibition of Unfair, Deceptive, or Abusive Acts or Practices (UDAAP). These regulations require that all consumers are treated fairly and that any adverse decisions are explainable. This has significant implications for the use of “black box” models, which can be difficult to interpret.

The following table provides a high-level overview of some of the key regulatory considerations for machine learning models in financial services:

Execution

The execution of a machine learning governance framework requires a systematic and disciplined approach. It is a continuous process of identifying, measuring, mitigating, and monitoring bias throughout the model lifecycle. This process should be embedded within the SI’s existing risk management framework and should be supported by a clear set of policies, procedures, and controls.

A key component of this process is the establishment of a dedicated model risk management function with the authority and expertise to oversee the development, validation, and deployment of all machine learning models. This function should be independent of the business lines and should report directly to senior management. This ensures that model risk is managed effectively and that the SI’s models are fair, transparent, and compliant with all relevant regulations.

Effective governance frameworks identify, assess, and mitigate risks such as biases, operational failures, and reputational damage, making AI systems robust and reliable.

A Practical Guide to Bias Mitigation

The following is a step-by-step guide to implementing a practical bias mitigation program:

1. Establish a Governance Framework The first step is to establish a clear governance framework for machine learning. This should include a model risk management policy, a set of standards for model development and validation, and a clear definition of roles and responsibilities.
2. Inventory and Risk-Tier Your Models The next step is to create an inventory of all machine learning models used by the SI and to risk-tier them based on their materiality and complexity. This will allow you to prioritize your governance efforts and focus on the models that pose the greatest risk.
3. Define Fairness Metrics It is essential to define a set of fairness metrics that can be used to measure bias in your models. These metrics should be tailored to the specific use case and should be aligned with your organization’s ethical principles.
4. Implement Bias Detection and Mitigation Techniques There are a variety of techniques that can be used to detect and mitigate bias in machine learning models. These include:
   • Pre-processing techniques These techniques are used to modify the training data to remove bias.
   • In-processing techniques These techniques are used to modify the learning algorithm to reduce bias.
   • Post-processing techniques These techniques are used to adjust the model’s predictions to ensure fairness.
5. Conduct Regular Fairness Audits It is essential to conduct regular fairness audits of your models to ensure that they are performing as expected and that they are not producing biased outcomes. These audits should be conducted by an independent team and the results should be reported to senior management.
6. Monitor and Report on Model Performance It is essential to continuously monitor the performance of your models and to report on their performance to senior management. This will allow you to identify any issues early on and to take corrective action as needed.

How Can We Quantify Model Fairness?

Quantifying model fairness is a complex but essential task. There are a number of different fairness metrics that can be used, each with its own strengths and weaknesses. The choice of which metric to use will depend on the specific context and the ethical considerations at play. The following table provides an overview of some of the most common fairness metrics:

Reflection

The governance of machine learning models within Systematic Internalisers is a complex and multifaceted challenge. It requires a deep understanding of the technology, the regulatory landscape, and the ethical implications of using AI in financial markets. However, it is a challenge that must be met if SIs are to maintain the trust of their clients and the integrity of the market. The frameworks and strategies outlined in this article provide a roadmap for developing a robust and effective governance program.

The ultimate success of this program will depend on the commitment of senior leadership, the expertise of the model risk management team, and the culture of the organization. A culture that values fairness, transparency, and accountability is the most effective defense against the risks of algorithmic bias.