Top 10 challenges affecting applications, APIs and gen AI

There are many challenges faced by organisations in their efforts to deliver and secure applications and application programming interfaces (APIs). In fact, the Open Worldwide Application Security Project (OWASP) has several lists which help organisations every day keep their applications, APIs, and now large language models (LLMs) secure from the incredibly robust array of attacks which threaten to disrupt business. However, no one to date has a top 10 list of challenges which threaten the delivery of applications, APIs, and, yes, generative artificial intelligence (gen AI).

Until now.

Application delivery may have started with the simple—but powerful—load balancing proxy, but it has evolved along with applications to incorporate a wide array of capabilities designed to ensure availability, enhance performance, and secure the increasingly important digital assets which power today’s internet economy.

Alongside the evolution of application delivery come a number of challenges affecting performance, availability and scalability. These affect an organisation’s ability to deliver and secure a hybrid, multicloud application and API portfolio, and ultimately deliver successful gen AI. So, what are some of these challenges?

1. Poor DNS practices

The Domain Name System (DNS) is a critical component of the internet’s infrastructure, translating domain names into IP addresses to route user requests to the appropriate servers. However, poor DNS practices can compromise application performance, availability, and scalability.

It can also significantly degrade application performance by increasing query response times and causing delays in resolving domain names. When Time-to-Live (TTL) settings, numerical values that indicate how long a data packet or record should exist on a network before it is discarded, are too low, DNS queries must be resolved more frequently. This increases the load on DNS servers and slows down application response time.

Additionally, improperly configured DNS servers or the lack of DNS security features like DNS Security Extensions (DNSSEC) can introduce delays by allowing unauthorised users to hijack or redirect traffic to slower or malicious servers.

Weak DNS practices can severely impact the performance, availability, scalability, and operational efficiency of applications. However, by implementing Domain Name System Security Extensions (DNSSEC), optimising TTL settings, and securing dynamic DNS updates, organisations can mitigate these risks and create a more reliable DNS infrastructure.

2. No fault tolerance

The lack of fault tolerance and resilience in application delivery strategies can lead to significant performance issues, reduced availability, and scalability limitations.

By implementing load balancing, failover mechanisms, and programmable infrastructure, organisations can create a more resilient system that supports continuous availability and optimal performance, even under challenging conditions. Emphasising fault tolerance enhances user experience, reduces operational overhead and supports efficient scalability - ensuring that applications can meet the demands of today’s fast-paced digital environment.

3. Insufficient observability

Observability is a critical aspect of modern application delivery, providing visibility into the health, performance, and usage of applications and infrastructure. Poor visibility becomes particularly problematic in complex environments, such as AI-driven applications, where real-time insights are essential.

Ultimately, incomplete observability in application delivery can lead to performance degradation, reduced availability, limited scalability, and operational inefficiencies. By implementing comprehensive monitoring and logging, adopting standardised observability with OpenTelemetry, and utilising dynamic alerting with automated responses, organisations can overcome these challenges.

4. Ineffective traffic controls

Effective traffic management is essential for delivering a seamless user experience, particularly as applications scale to support larger audiences and more dynamic workloads. However, insufficient traffic controls can lead to issues like overloading backend services, susceptibility to Distributed Denial of Service (DDoS) attacks, and inefficient resource usage.

By implementing rate limiting, throttling, and caching mechanisms, organisations can manage traffic more effectively, prevent service disruptions, and support scalable growth.

Emphasising robust traffic management practices is essential for delivering high-performance, resilient applications that can adapt to changing user demands and provide a consistent experience across diverse environments.

5. Insufficiencies impacting traffic steering

Unoptimised traffic steering—caused by static routing policies, lack of dynamic decision-making, or insufficient load-balancing algorithms—can lead to performance bottlenecks, inconsistent availability, and limited scalability.

In AI-driven applications, where processing needs can vary based on data types and user demand, efficient traffic steering is essential for maintaining responsiveness.

By adopting best practices such as dynamic routing, intelligent load balancing, and programmable ADCs, organisations can optimise traffic flows, improve resource utilisation, and ensure that applications meet variable demand.

6. Unmanaged latency

Latency is a key factor affecting application delivery, particularly in data-intensive environments like AI applications. The inability to handle latency effectively can lead to performance issues, reduced availability, and limited scalability, especially as applications grow and user demands fluctuate. Latency bottlenecks result from various issues, such as suboptimal data routing, inefficient processing, and inadequate resource allocation.

By implementing optimised data routing, edge computing, and adaptive resource allocation, organisations can mitigate latency challenges and support a high-performance, resilient infrastructure.

7. Incompatibility across hybrid cloud environments

In hybrid multicloud environments, incompatible delivery policies can pose significant challenges, and can lead to soaring operational overheads. Incompatibilities of this nature often arise when organisations use multiple cloud providers, each with unique traffic routing, security, and data handling protocols.

According to LoadView, a leading cloud-based load testing platform, applications with inconsistent delivery policies across multiple regions experience 50% more latency in cross-border data transfers than those with region-specific optimisations.

By standardising metrics, aligning service capabilities, and leveraging programmable infrastructure, organisations can overcome these challenges.

Emphasising consistency and flexibility in delivery policies ensures that applications can maintain high performance, availability, and scalability across a hybrid multicloud infrastructure.

8. Failing to meet regulatory requirements

As governments worldwide enforce stricter laws on data sovereignty, security, and privacy, regulatory compliance has become essential. Organisations failing to meet these regulations expose applications to security vulnerabilities and introduces performance bottlenecks and scalability constraints. These challenges are particularly prevalent in AI-driven applications.

By implementing strong encryption, utilising Federal Information Processing Standards (FIPS)-compliant devices, and adopting automated compliance tools, organisations can address these risks and support secure, scalable, resilient and compliant application delivery.

9. Standard infrastructure can’t support unique requirements

As digital applications become increasingly specialised, organisations are often faced with unique requirements that standard infrastructure cannot support.

Programmability within the application delivery infrastructure offers a powerful solution to such challenges, enabling organisations to tailor their infrastructure to support complex, customised requirements.

Bespoke application requirements often challenge traditional application delivery solutions, as they require customisation which standard infrastructure cannot provide. By leveraging programmability within the application delivery infrastructure, organisations can adapt to these unique demands, ensuring high performance, availability, and scalability.

Furthermore, programmable infrastructure enables seamless transitions, integrates new services efficiently, and supports custom load balancing, allowing organisations to deliver reliable and responsive services which meet the specific needs of their users.

10. Resource inefficiency

Many organisations struggle with resource inefficiencies due to mismatched distribution algorithms or inadequate health check mechanisms. These inefficiencies can lead to wasted compute power, increased operational overhead, and strained infrastructure, ultimately impacting performance, availability, and scalability.

By leveraging programmability, intelligent health checks, and dynamic traffic steering, organisations can optimise resource usage, improve application performance, and enhance scalability.

Facing application delivery challenges head on

Boosting performance, availability and scalability in application delivery, APIs and gen AI is full of challenges. By taking meaningful steps to address these problems, organisations will be better placed to improve API success and scale efficiently as business needs evolve.