On Security and AI Platforms. Q&A with Paul Speciale

Q1. GPUs, with their ability to process vast datasets in parallel have become the backbone of AI workloads. What are these GPU-accelerated architectures to accommodate complex models and massive datasets?

Paul Speciale: GPU-accelerated architectures today have terabytes of high-speed flash internal memory, high-speed networking interfaces. This should be pared with scalable, disaggregated storage systems capable of delivering extremely high data throughput. To support massive AI datasets, storage systems should be also able to scale horizontally and deliver low-latency access to large volumes of data. File systems such as POSIX-compliant storage integrated with technologies like GPU Direct help eliminate CPU bottlenecks, improving computational efficiency. More recently, object storage in on-premises, cloud and hybrid environments, is becoming a key solution due to its dramatic scalability, reduced overhead, and better handling of unstructured data. This flexibility allows AI workflows to scale seamlessly across distributed resources, addressing the storage and compute demands of modern AI models.

Q2. What if you do not have access to such GPUs?

Paul Speciale: Data preparation phases can leverage standard CPU architectures, but for compute-intensive model training, fine-tuning and inferencing processing – access to high-performance GPUs is really essential or AI workloads will face significant bottlenecks. Looking at the AI workflow pipeline, model training and inference is where massive computational power is required to process large datasets in parallel. In addition, enterprises are often forced to rely on legacy compute and storage infrastructure, which will be inefficient and slow, leading to extended training times and reduced scalability. Moreover, without GPUs, the workload may shift to CPU-based processing, which is not optimized for the matrix-heavy operations common in AI models. This can lead to longer processing times and higher costs due to the need for more physical infrastructure and energy consumption.

Q3. While enterprises adopt direct GPU access to support high-throughput AI training and inference, the security landscape grows more complex. Do you agree with this?

Paul Speciale: Yes, absolutely. Technologies like GPU Direct and direct access to GPUs provide significant performance improvements but also open up new vectors for potential vulnerabilities. For instance, the open shared memory architecture inherent in GPU Direct can create avenues for data leakage if not properly secured. Additionally, direct memory access (DMA) techniques, though efficient for data throughput, can be exploited for malware injection through buffer overflows, especially if security controls are lax. Cross-tenant vulnerabilities are also a concern, particularly in cloud or virtualized environments where isolation may not be robust enough to prevent malicious workloads from accessing resources or data of other tenants. As such, ensuring secure configurations, access controls, and encryption is crucial to balancing performance gains with security.

Q4. It is said that the default assumption is that compute-intensive environments like HPC and cloud-native AI platforms come with baked-in security. Is this true?

Paul Speciale: No, this is a common misconception. While cloud-native AI platforms and HPC environments often feature sophisticated infrastructure, they do not inherently come with built-in security. These environments can involve complex hardware and software stacks, where vulnerabilities may arise from shared resources, virtualization layers, or else improperly configured access controls. In particular, the dynamic and scalable nature of cloud environments can inadvertently expose sensitive data if not managed properly. Furthermore, the decentralized nature of many cloud architectures, especially with GPU access, necessitates robust and continuous monitoring, alongside layered security protocols such as identity-based access control, encryption, and network segmentation, to ensure security is not compromised.

Q5. GPU Direct and similar technologies accelerate AI workloads significantly, but they also open the door to a new class of risks. Which ones?

Paul Speciale: Direct GPU access technologies can enhance AI workload performance but also introduce notable risks. These include the potential for data leakage via shared GPU memory, where sensitive data could be accessed by unauthorized users or processes. Additionally, direct memory access (DMA) exposes systems to potential malware injection through memory buffer overflows. In multi-tenant environments, where multiple workloads share GPU resources, the lack of strong isolation mechanisms increases the risk of cross-tenant attacks, where a compromised tenant could access or corrupt the data of others. The high throughput nature of GPU Direct can make it more challenging to monitor and secure workloads in real time, further complicating threat detection. To mitigate these risks, advanced isolation techniques, regular patching, and robust encryption must be employed.

Q6. You mentioned Data Leakage, Malware Injection and Cross-Tenant Exploits. How serious are these risks? What is the probability that they occur? 

Paul Speciale: These risks are very serious, especially in cloud or multi-tenant environments, where workloads and sensitive data are more exposed. Data leakage can lead to substantial privacy violations and regulatory issues, particularly when dealing with personally identifiable information (PII) or proprietary datasets. Malware injection risks pose threats to system integrity, potentially causing disruptions or even irreversible damage to both the application and infrastructure. Cross-tenant exploits are particularly concerning because they can compromise the isolation of workloads in shared environments, leading to cascading attacks across different tenants. The probability of these risks materializing increases with the complexity and scale of the infrastructure, especially if adequate security controls—such as robust encryption, data isolation, and multi-factor authentication—are not in place. Regular vulnerability assessments and proactive threat monitoring can help reduce these risks.

Q7. Why are these vulnerabilities especially concerning in cloud environments where hardware is virtualized and shared?

Paul Speciale: Cloud environments, by their nature, expose users to a shared hardware model, which inherently increases the attack surface. Virtualization, while offering flexibility and scalability, can also obscure direct access to physical hardware, making it harder to detect or prevent unauthorized access. The shared GPU infrastructure commonly used in AI workloads compounds this issue, as vulnerabilities such as poor tenant isolation or insufficient encryption of data in use could in theory allow attackers to exploit memory buffers or other shared resources. Additionally, the complexity of cloud-based architectures often leads to misconfigurations in access control, further exacerbating security risks. Continuous monitoring, automated compliance checks, and endpoint security systems are vital in such environments to detect and mitigate potential threats.

Q8. Multi-tenancy introduces a fundamentally different threat model. Which one?

Paul Speciale: Multi-tenancy invites  a cross-tenant threat model, which differs from traditional single-tenant models by introducing risks that arise from within the shared infrastructure itself. In this model, malicious or compromised tenants can leverage shared resources to attack other tenants, accessing or corrupting their data. The risks extend beyond simple unauthorized access to include potential disruptions, data manipulation, or even denial of service to other tenants. As such, a multi-tenant model requires more stringent isolation policies, tenant-specific access controls, and enhanced monitoring tools that can detect anomalous behaviors and prevent cross-tenant security breaches.

Q9. What are your recommendations to mitigate all of these risks?

Paul Speciale: Mitigating these risks requires a robust security strategy that incorporates multiple layers of protection. First, granular, identity-based access controls should be enforced across compute, storage, and networking resources to ensure that only authorized entities can access critical data and infrastructure. Comprehensive encryption—covering data at rest, in transit, and, where feasible, during computation—should be applied to protect the confidentiality and integrity of sensitive information. In addition, leveraging Trusted Execution Environments (TEEs) can provide additional protection for data during processing, preventing unauthorized access even within potentially compromised systems. Adopting software-defined storage architectures with built-in resilience features, such as immutability, Write Once Read Many (WORM) capabilities, and anomaly detection, can dramatically enhance data security and integrity. Finally, employing secure-by-design object storage systems with built-in telemetry, real-time threat detection, and automated recovery mechanisms ensures operational integrity, enabling continuous data availability while minimizing the impact of potential security incidents.

………………………………………………………..

Paul Speciale, CMO Scality

Over 20 years of experience in Technology Marketing & Product Management. Key member of team at four high-profile startup companies and two fortune 500 companies.

Sponsored by Scality