Secure AI Agent Development: Trends and Challenges

In the rapidly evolving landscape of artificial intelligence (AI), the development of AI Agents has become a focal point for enterprises… nearly all of them. According to recent IBM research, 99% of respondents are exploring or actively developing AI agents. This surge in interest also serves to underscore the necessity for secure AI agent development. 

This blog dives into the importance of AI Security Posture Management (AISPM) and how as AI agents are developed en masse, security teams need to implement controls around how they are built, and the ensuing runtime risks that occur. Further, we will delve into platforms like Microsoft Copilot Studio and Salesforce Agentforce, which enable anyone to build AI Agents without needing to be a professional developer, the associated security risks, ahd what security leaders can do about them.

The Rise of AI Agent Development

AI agents are transforming various business functions, from customer support to human resources. IBM’s survey highlights that the most common use cases for AI agents include customer service/support, project management, personal assistants, and content creation. These agents are designed to handle repetitive tasks, provide quick responses to customer inquiries, and assist in managing workflows, thereby enhancing efficiency and productivity.

For instance, in customer support, AI agents can manage high volumes of inquiries, providing instant responses and freeing up human agents to handle more complex issues. In human resources, AI agents can streamline recruitment processes by screening resumes and scheduling interviews, allowing HR professionals to focus on strategic initiatives.

The Wall Street Journal also recently published an article further illustrating the diverse applications of AI agents. Companies like Johnson & Johnson are using AI Agents to help with pharmaceutical drug discovries, while Deutsche Telekom uses AI Agents for its 80,000 German employees to ask questions about internal policies and benefits. 

How AI Agents Are Developed

Low-code tools have revolutionized the way AI agents are developed. These tools enable developers to create sophisticated AI agents with minimal hand-coding, making the development process faster and more accessible. According to IBM’s survey, 65% of developers are actively using low-code tools, while 59% are using no-code tools to build AI agents. 

Microsoft Copilot Studio and Salesforce Agentforce are leading the charge in providing robust low-code platforms for AI agent development, but not just for professional developers, but citizen developers as well. These platforms offer intuitive interfaces and pre-built components that simplify the development process, allowing developers to focus on creating innovative solutions rather than getting bogged down by complex coding tasks, and for business users to bring their real-world knowledge and expertise into previously closed off development cycles, to build agents without needing to make requests to IT or developer teams, thus freeing them up to do other important tasks.

The Necessity of Low-Code Tools

The necessity of low-code tools in AI agent development cannot be overstated. With only one-third of respondents willing to invest more than two hours in learning a new AI development tool, the demand for user-friendly, efficient development platforms is clear. Low-code tools cater to this need by providing a streamlined development experience that reduces the learning curve and accelerates time-to-market.

This democratization of AI development fosters innovation and allows organizations to leverage the diverse skills of their workforce. However, it also introduces a host of risks that security teams need to be mindful of.

Prevalence of Low-Code Tools: Microsoft Copilot Studio and Salesforce Agentforce

Microsoft Copilot Studio and Salesforce Agentforce exemplify the prevalence and effectiveness of low-code and no-code tools in AI agent development. Microsoft Copilot Studio integrates seamlessly with the Microsoft ecosystem, providing developers with a comprehensive suite of tools to build, test, and deploy AI agents and integrate them not only as standalone agents and bots, but as extensions of Copilot for M365 to extend business productivity apps. Its low-code environment enables rapid prototyping and iteration, making it an ideal choice for enterprises looking to stay ahead in the AI race.

Similarly, Salesforce Agentforce offers a powerful low-code platform tailored for customer service and support applications. With its drag-and-drop interface and extensive library of pre-built components, Agentforce simplifies the creation of AI agents that can handle a wide range of tasks, from answering customer queries to managing workflows.

Security Risks in Low-Code AI Agent Development

While low-code tools offer numerous benefits, they also introduce unique security risks. IBM’s research highlights several concerns, including the trustworthiness of AI agents, the risk of agents being compromised by malicious actors, and the challenge of adhering to compliance and regulations. Zenity Labs has also put forth research, in Q4 of 2024, highlighting common risks that stem from these low-code tools, which evade traditional security controls, software development lifecycles, and CI/CD tooling. 

One of the predominant risks that comes from these tools is over-sharing Agents that contain access to sensitive data. Because business users are put in charge of making decisions about what knowledge Agents have and what actions they can take, oversharing Agents can become a big risk without proper security guardrails in the development cycle. 

Additionally, the rapid development cycle enabled by low-code tools can sometimes result in insufficient testing and validation. This can lead to the deployment of AI agents that do not adhere to corporate security policies or regulatory requirements. Ensuring transparency and traceability of data is crucial in mitigating these risks.

Addressing Security Concerns

To address these security concerns, enterprises must implement an AISPM program that ensures the ethical and trusted lifecycle of AI agents. This includes establishing clear guidelines for data transparency and traceability, understanding what knowledge sources, triggers, actions, and other Agents are incorporated, as well as rigorous testing and validation processes for Agents that are being built across the enterprise.

Moreover, organizations should invest in training and upskilling their workforce to ensure that developers, whether professional or citizen, are equipped with the knowledge and tools to build secure AI Agents. This includes understanding the business objectives and outcomes of AI projects, as well as the specific security requirements that must be met.

Conclusion

The development of secure AI Agents is a critical priority for enterprises, and low-code tools play a pivotal role in this process. Platforms like Microsoft Copilot Studio and Salesforce Agentforce are leading the way in providing accessible, efficient development environments that empower developers to create innovative AI solutions. However, it is essential to address the associated security risks through robust governance and comprehensive training. By doing so, organizations can harness the full potential of AI agents while ensuring their trustworthiness and compliance with regulatory standards.

In conclusion, the integration of low-code tools in AI agent development is not just a trend but a necessity. As enterprises continue to explore and develop AI agents, the focus must remain on creating secure, reliable, and compliant solutions that drive innovation and business success.