ChatGPT and Claude versus a local AI agent: hosted entirely in Germany on its own server

The short answer: the data class of a task, not the league table of models, decides the operating model. ChatGPT and Claude offer the highest performance and, in their enterprise plans, dependable commitments on training, ownership of outputs and data storage[1][2]. A local AI agent hosted entirely in Germany on its own server offers a different jurisdiction, its own memory and operational control, but falls short on peak performance[3]. In our assessment, many mid-sized companies are therefore best served by running two tracks, governed by a policy on data classes.

Disclosure: Avolang itself operates a local AI agent with Alwisr Company OS and therefore has a commercial interest in the outcome of this analysis. The article draws on public studies and vendor statements, names the limits of our own approach and marks our own assessments as such. All information is as of 29 October 2026. It does not replace legal advice.

Starting point: usage is growing faster than governance

Artificial intelligence (AI) has reached everyday business operations, and with it the question of the right operating model. In its Applied AI Index 2026, a survey of 1,330 executives, Boston Consulting Group (BCG) classifies 7.5 percent of companies as “future-built”. A further 41 percent are actively scaling AI and achieve, for example, 1.8 times the shareholder return of the laggards[10]. A year earlier, BCG had put 5 percent in the “future-built” group, 35 percent in the scaling group and 60 percent among laggards with minimal returns[11]. The annual State of AI survey by McKinsey & Company, reported by FM Magazine on 1 September 2026, paints a more sober picture. Only about 6 percent of respondents qualify as “high performers”; they attribute at least 5 percent of their earnings before interest and taxes (EBIT) to AI and describe its impact as significant. At least some EBIT effect is reported by 37 percent[12].

Governance is not keeping pace. According to BCG, AI spending has doubled within a year, from about 1.7 percent to 3.3 percent of revenue, and more than 80 percent of it sits outside the IT budget. Some 42 percent of companies expect to give AI agents real decision-making authority by 2030, yet only 5 percent have the full controls in place for it today. In BCG’s view, the binding constraint is now whether companies can control and steer AI[10].

A survey of 604 companies with 20 or more employees, which the digital association Bitkom commissioned in summer 2025, shows what this gap looks like in daily practice. In 8 percent of the companies, private AI tools such as ChatGPT are widely used (2024: 4 percent), and a further 17 percent report isolated cases. Access to generative AI is provided to employees by 26 percent of companies, and 23 percent have set rules for its use[13]. An indication of the financial dimension comes from the Cost of a Data Breach Report 2025 by IBM and the Ponemon Institute: organisations with a high share of shadow AI, that is, unapproved AI use, recorded on average US$670,000 higher costs per data breach than those with little or no shadow AI. The sample comprises 600 organisations that experienced a data breach and is therefore not a cross-section of all companies[14].

For the operating-model question, this means that the choice between ChatGPT, Claude and a local agent is first a question of governance and only then a question of product (Avolang assessment). Bitkom advises companies to avoid uncontrolled proliferation, to set clear rules and to provide AI technologies themselves[13]. We describe the framework for such a policy in our guide to an AI policy and in the article on AI governance in business; the liability questions raised by shadow AI are covered in our white paper.

Frontier models: peak performance and dependable enterprise contracts

Frontier models, that is, the most capable language models of the large providers, remain the reference point. The AI Index 2026 of the Stanford Institute for Human-Centered AI describes a tight leading group for March 2026: Anthropic (1,503), xAI (1,495), Google (1,494) and OpenAI (1,481) are within 25 Elo points, a rating measure borrowed from chess, here based on human votes on the Arena Leaderboard. Stanford concludes that competitive pressure is shifting to cost, reliability and domain-specific performance[3]. For the choice between ChatGPT and Claude, this means that no single league table settles the decision. Integration, contract terms and results on one’s own tasks matter more (Avolang assessment).

A gap to open models persists. The best closed model led the best open model by 3.3 percent in March 2026; in August 2024 the lead was 0.5 percent[3]. Epoch AI puts the lag of the most capable open models on the Epoch Capabilities Index, an aggregated performance measure, at an average of four months since January 2026, and at six months under a stricter definition[15]. For tasks at the performance frontier, such as complex analysis, demanding programming or very long documents, frontier models are in our assessment the first choice.

Contract terms add to this. For the business offerings of ChatGPT and Claude, they go well beyond the consumer product. OpenAI by default does not train its models on data from the ChatGPT business plans and the application programming interface (API) unless a customer explicitly agrees. Since February 2025, OpenAI has offered data residency in Europe, that is, the storage and processing of data in a chosen region: new workspaces of ChatGPT Enterprise and Edu can store content in Europe, and for new API projects Europe can be selected as the region. In January 2026, options were added for processing on graphics processing units (GPUs) in Europe or the United States for eligible ChatGPT Enterprise and Edu customers[1]. In its Commercial Terms, Anthropic commits that the customer retains the rights to its inputs, that outputs belong to the customer and that Anthropic may not train models on customer content. Where third parties claim that paid use infringes their intellectual property, Anthropic indemnifies the customer[2]. These commitments relate to the commercial products; separate terms apply to consumer plans[5]. Private accounts of employees, as Bitkom describes them, are not covered by the enterprise contracts.

The providers now also offer a memory. Anthropic has introduced a memory for Team and Enterprise plans that is separated by project; administrators can edit memories, and subscribers to Team plans can import and export memories[7]. For ChatGPT, OpenAI describes functions to view, correct and delete stored memories, but notes that functions and controls vary by plan, region, platform and workspace[8].

In summary, the case for frontier models rests on:

• the highest available model performance across a broad range of tasks
• immediate usability without in-house operation and without an integration project
• contractual commitments on training, ownership and, at OpenAI, data residency in Europe
• indemnification for third-party intellectual property claims, at least at Anthropic
• continuous development of the models by the provider

Data protection with ChatGPT and Claude: jurisdiction and retention as open points

Contractual commitments on training leave one question unanswered: in which jurisdiction does the provider sit? For passing personal data to a service provider, the General Data Protection Regulation (GDPR) requires a data processing agreement under Article 28; for transfers to a third country, Articles 44 to 49 apply[16]. For transfers to the United States, the EU-U.S. Data Privacy Framework comes into consideration. The European Commission adopted an adequacy decision on it on 10 July 2023; personal data may accordingly flow freely to US companies that participate in the framework[17][18].

The General Court of the European Union dismissed the action against this decision on 3 September 2025 and confirmed that the United States ensured an adequate level of protection at the time of the decision[18]. An appeal against the judgment has been lodged with the Court of Justice (Case C-703/25 P)[19]; to our knowledge, it had not been decided as of 29 October 2026. We offer no forecast of the outcome. For planning purposes, this means that the framework applies, but its legal standing is not beyond dispute.

Independently of this, the question of access by US authorities remains. Under 18 U.S. Code § 2713, which the US CLOUD Act (Clarifying Lawful Overseas Use of Data Act) inserted in 2018, providers of electronic communication services and of remote computing services must comply with the obligations of the relevant chapter to preserve, back up or disclose the contents of communications and any records or other information pertaining to a customer or subscriber that are within their possession, custody or control, whether located inside or outside the United States[6]. The European Data Protection Board and the European Data Protection Supervisor addressed the effects of the Act on the European legal framework as early as 2019, in a joint response to the Committee on Civil Liberties, Justice and Home Affairs of the European Parliament[20]. In our assessment, it follows that storage in Europe changes the storage location, not automatically the provider’s legal jurisdiction. Whether a specific disclosure request would be lawful, and how it relates to the GDPR, is a legal question to be decided case by case.

Retention obligations arising from court proceedings come on top of this. OpenAI commits to removing deleted ChatGPT conversations from its systems within 30 days unless it is required to retain them[21]. How far such obligations can reach is shown by an order of a US court in the proceedings brought by the New York Times. According to OpenAI, it covered the ChatGPT Free, Plus, Pro and Team plans as well as the API without a zero-data-retention agreement, but not ChatGPT Enterprise and Edu[21]. The choice of plan is therefore also a data protection decision.

These points do not argue against the use of frontier models. They determine for which data such models are suitable. The assessment in each individual case belongs with data protection and legal advisers.

The local AI agent: entirely in Germany, on its own server, with its own memory

Agents are no longer a fringe topic. BCG expects agentic AI to account for about 40 percent of total AI value by 2030[10]. According to McKinsey, 40 percent of large companies are already scaling AI agents in at least one function (prior year: 27 percent), while the figure for smaller organisations is unchanged at 22 percent[12].

A local AI agent is an AI assistant operated for a single company: on servers in a jurisdiction the company chooses, with its own memory and with approval before every action. “Local” here refers to location and allocation, not to an individual workstation: anyone who wants to run an AI locally does not have to install it in their own building. The agent runs on servers in Germany, separate from other customers, and not at an AI provider abroad. The term sovereign AI is also used for systems in which a company determines jurisdiction, operation and data storage itself.

Avolang offers such an agent with Alwisr Company OS. According to the product description, the system runs entirely in Germany; each company receives its own, isolated area on servers in Germany. Avolang does not use customer content for training, and the models run at Avolang or at the customer, not at an AI provider abroad. The system remembers content across all conversations. Sending, deleting, changing and executing commands happen only after a click or within an approved automation, and new automations first run for seven days in shadow mode, that is, without sending or changing anything[4].

The product description distinguishes three operating options, all of which run in Germany[4]:

• Hosted by Avolang. The agent runs in a German data centre, in a separate area for the company. This is the lowest-cost option and the fastest way to start.
• Dedicated server. The agent runs on a server in Germany that is reserved for the company alone and separate from all other customers.
• On site, on the company’s own hardware. The agent and the models run on hardware at the company. That needs powerful servers and is the most expensive option.

According to its own information, Avolang operates the hosted option in the IONOS data centre in Germany, with language models that are considerably smaller than the frontier models. Running one’s own models on one’s own servers is technically established: frontier providers also publish open models. OpenAI describes gpt-oss-120b as a high-performing open model that fits on a single H100 graphics processor and makes it available under the Apache 2.0 licence[22]. How operating on site, hosted operation and hybrid solutions can be weighed against one another is described in the article on local IT infrastructure.

That small models can be sufficient for many tasks is also argued outside the vendor market. In a position paper, NVIDIA Research argues that small language models are powerful enough, more suitable and more economical for many calls in agentic systems; where general conversational ability is essential, systems that invoke several different models are the natural choice[23]. In April 2025, Gartner predicted that by 2027 organisations will use small, task-specific models three times more often than general-purpose language models[24]. Both are positions and forecasts, not measurements in an individual company. Whether a small model solves one’s own tasks can be decided only with one’s own test tasks.

The limits of the local approach are equally important to state:

• Lower peak performance. Even the most capable open models trail by four months on average[15]; in our assessment, smaller models trail further and do not reach the level of frontier models on complex reasoning and demanding texts.
• Setup and integration effort. An agent is useful only once it is connected to mailboxes, document repositories and line-of-business systems. That takes project work that an instantly usable chat does not need.
• Data quality. A memory is only as good as the content it holds. Outdated or contradictory documents lead to outdated or contradictory answers (Avolang assessment).
• Dependence on the operator. Whoever outsources operation replaces dependence on a US provider with dependence on a service provider. Data return and export of the memory therefore belong in the contract (Avolang assessment).
• Operational responsibility. Security updates, monitoring and recovery must be regulated, whether with the service provider or in-house (Avolang assessment).

Memory: six requirements for an AI agent with its own context

Memory is where an agent differs most from a chat in daily use: it knows customers, cases and decisions the next day as well. At the same time, it is the most sensitive part under data protection law, because knowledge about people and business accumulates there. From the documented functions of the frontier providers, we derive six requirements that an AI memory should meet regardless of where it is operated (Avolang assessment). They describe requirements, not product commitments; which of them a specific solution meets has to be checked case by case.

1. Separation by ownerMemories are separated by company, team or project so that confidential content does not enter foreign contexts. Anthropic names these project boundaries expressly as a safeguard[7].
2. TraceabilityEvery answer shows which memories or documents it relies on. For personalised answers, OpenAI displays sources, but notes that these do not show every influencing factor[8].
3. Inspection and correctionStored content can be viewed, corrected and deleted, by users and by administrators[8][7].
4. Retention and deletionFixed periods apply to each type of memory, and deletion also takes effect in backups. The reservation of statutory retention obligations, which even OpenAI names for deleted conversations, has to be observed[21].
5. Export and portabilityThe memory can be exported in an open format so that a change of provider does not start from zero. Anthropic allows the import and export of memories in Team plans, but describes the import as experimental and still in development[7].
6. Targeted retrieval instead of permanent contextOnly relevant entries are retrieved, for example through a combination of keyword and semantic search, and the full memory is not loaded into every request. A 2023 study shows that models use information from the middle of long inputs less well than information at the beginning or end, including models designed for long contexts[25]. How pronounced this effect is in current models has to be tested case by case.

How such a memory builds on company knowledge is shown in the comparison of enterprise search, knowledge base and AI assistant.

Side-by-side comparison: seven criteria at a glance

Exhibit 1 compares the two operating models. Source references appear in the cells; statements without a reference reflect Avolang’s assessment.

Exhibit 1: Frontier models and local AI agent compared

Cost of a local AI agent: the order of magnitude of model costs

When looking at cost, it pays to separate the components: setup, platform operation, integration, data maintenance, support and model usage. With hosted open models, model usage is usually small relative to the other items (Avolang assessment). Exhibit 2 calculates the order of magnitude for three task profiles, using the public list prices of the IONOS Cloud AI Model Hub[9]. Billing is per token, the text units into which a model splits inputs and outputs. Profile C describes an agent run in which the growing context is passed to the model again at every work step; the assumptions behind the profiles come from Avolang. On a dedicated server, billing per token no longer applies; the compute capacity provided takes its place (Avolang assessment).

Exhibit 2: Model costs per 1,000 tasks in euros (list prices, as of 29 October 2026)

List prices per 1 million tokens (input / output): Mistral Small 24B Instruct €0.10 / €0.30, gpt-oss-120b €0.15 / €0.65, Qwen3.5-397B-A17B €0.60 / €3.60[9]. Values rounded to two decimal places. Avolang’s own calculation based on the price page, retrieved on 29 October 2026. The price page states no value-added tax; the amounts are taken over as listed. Volume discounts and minimum commitments are not taken into account. Setup, platform operation, integration, data maintenance and support are not included. gpt-oss-120b has 117 billion parameters, of which 5.1 billion are active[22].

Even the largest of the three models causes €261.36 of model costs at 2,200 agent runs a month, the smallest €38.28. In practice, the cost drivers are setup, integration, data maintenance and operation (Avolang assessment). The figures show the order of magnitude of model costs for hosted open models, not the price of Alwisr. With Alwisr, setup is fixed in writing in advance at a fixed price, operation costs a monthly amount depending on the variant, and the model costs are stated in the quote[4].

The prices of the frontier providers are not part of this calculation. They depend on model, plan and usage pattern, and they change frequently. A dependable comparison requires calculating one’s own task profiles against the current price lists of both sides.

Recommendation: two tracks, governed by data classes

The comparison does not lead to an either-or decision. A sensible approach is to operate in two tracks governed by a policy on data classes: non-critical tasks run through enterprise accounts with frontier models, personal and confidential tasks through a local agent in Germany. The two tracks are separate accesses and are not technically linked; Alwisr uses models on servers in Germany and does not call any AI provider abroad[4]. This has a price: employees must know which track applies to which data (Avolang assessment). The framework for the associated policy is described in the guide to an AI policy.

Exhibit 3: Data classes and recommended track (Avolang assessment)

Five decision questions for management

1. Which data classes arise?Without classification, no track can be assigned. Who defines the classes, and who checks compliance?
2. Which tasks recur?Recurring tasks that draw on company knowledge justify integration and memory; one-off tasks usually do not.
3. What performance is actually needed?The answer comes from your own test tasks, not from league tables.
4. What does the exit look like?Data return, export of the memory and contract termination have to be clarified before signing, for both tracks.
5. Who bears operation and responsibility?Internally, with the service provider or shared: the answer determines roles, response times and liability.

In 90 days to a viable operating model

Companies that want a local AI agent built should proceed in stages. In June 2025, Gartner predicted that more than 40 percent of agentic AI projects will be cancelled by the end of 2027[26]. A staged entry with clear measurement points reduces this risk (Avolang assessment).

Frequently asked questions about ChatGPT, Claude and the local AI agent

Conclusion

Frontier models and a local AI agent in Germany solve different problems. Frontier models offer the highest performance and dependable enterprise contracts; on jurisdiction and control of the memory, however, companies remain dependent on the provider’s commitments. A local agent offers jurisdiction, memory and approvals under the company’s own control, but does not reach peak performance and requires setup, data maintenance and a reliable operator. In our assessment, the path for many mid-sized companies leads through a policy on data classes, two separate tracks and a staged entry with their own test tasks. After 90 days, this yields a dependable basis for decision that rests on data and on the company’s own results rather than on a product preference.