Questions to ask vendors about their AI security offerings: There are several areas where CISOs will want to focus their attention when considering AI-powered cyber solutions, including the following:Shadow AI: Uncovering and addressing shadow AI throughout the organization is a key issue for security leaders today. But so too is ensuring that sanctioned AI-enabled solutions are not misused in similar ways.To protect against this, CISOs should ask: Does the vendor offer discovery capabilities to help identify shadow AI usage? What policies and procedures, education and training, identity management and access control, data leakage protection, does the vendor offer to enable employees to continue to use AI, while layering on security features?Data protection: The superpower of AI security tools is their ability to ingest and process vast amounts of data in near real-time. But where does that data reside? On-prem, in the cloud, or both? Who is responsible for protecting LLMs and other data stores both at rest and in motion? If the vendor is using homegrown or third-party AI models, and providing conduits to the customer’s third-party AI models, how does the vendor protect those pipelines? How does the security team detect vulnerabilities or data leakage in a “black box” LLM? Who is responsible for protecting LLMs against prompt injection attacks or other types of model manipulation? Will my data be used to train the vendor’s models and, if so, how can I be sure that data is protected?Metrics: Much of the initial hype surrounding AI has turned to disappointment because organizations are struggling to identify benefits from AI pilot projects. CISOs need to be able to provide measurable results for any AI security tool. That can include improved mean-time-to-discovery (MMTD) and mean-time-to-recovery (MTTR) in the event of a breach, a quantifiable reduction in the rate of false positives, improved productivity among SOC staffers, increased accuracy of anomaly detection and threat hunting activities. CISOs should ask vendors and advisors, What metrics will best reflect the value these AI capabilities will have, and can those be captured to help assess the efficacy of the AI capabilities and our use of them? Workforce: What kind of training does the vendor provide for the most efficient use of AI, generative AI, and most importantly, agentic AI. Will the AI tool be able to automate low-level tasks so that my SOC analysts can focus on higher-level activities? How does the vendor offering help me to address the skills gap? Are there models and best practices for reorganizing my workforce for the era of AI. Will the use of AI security tools help address overwork and burnout among my staff? Are there specific guidelines or best practices for how my security team should interact with the AI in a human-in-the-loop, copilot-style scenario?Integration: How will the AI security tool integrate with my current security stack and my security processes and procedures? Most CISOs already have an overload of tools, EDR, XDR, SIEM, SOAR, CSPM, etc. What APIs and pre-built connections are provided to seamlessly integrate with my existing infrastructure? What types of agreements and alliances do you have with other vendors? How can I maintain a single dashboard? If a platform vendor has recently acquired an AI security tool, how well is that new capability integrated within the platform? Regulation: How will your tool conform to the specific regulatory requirements in my industry with regard to data storage and data privacy. Do you keep up with changes to regulations?Trust: How can I make sure that my security team trusts the decisions and recommendations that the AI systems make? In what ways can security practitioners go back and retrace how a model reached a conclusion?Scalability: It’s to be expected that data stores will increase in volume. And an enterprise might pilot the tool at one location, with plans to roll it out globally. How can I be sure the cloud-based AI tools can scale to meet my needs? Can the system handle large traffic volumes without performance delays. Does the solution encompass endpoints, networks, cloud, SaaS?Roadmap: What is your roadmap for updating the tool, delivering timely patches and providing new capabilities on a regular basis?Model integrity: How do you address concerns about bias in your models. How do you ensure data accuracy and integrity? How do you assure that the models are constantly updated to reflect changing real-world conditions.Vendor credibility: How long has this vendor been around? Do they have leaders with an industry pedigree? Do they have references that you can check? What is the financial viability of the company? For a startup, how much money have they raised? Are they generating revenue?Cost: What are the licensing terms? What types of SLAs or other performance metrics are included with a subscription?
First seen on csoonline.com
Jump to article: www.csoonline.com/article/4094763/key-questions-cisos-must-ask-before-adopting-ai-enabled-cyber-solutions.html
