Engineering in the age of AI

You have decided on AI. Now the choices that decide success.

Dr.-Ing. Lena Golgolnia · 21 October 2026 · 11 min read

In short

Adoption is broad, value is narrow: only 37% of organisations attribute any EBIT impact to AI, and about 6% are high performers.¹
Analysts point to the same causes: unready data, weak risk controls, rising costs and unclear value.⁴
Once the decision for AI is made, five choices decide success: the workflow, the kind of AI, the architecture and data, evaluation and oversight, and governance.
There is no universally best solution. The most capable or “smartest” option is often not the right one; the right one fits the risk, data, team and regulation of the case.
EU rules are part of the design: AI literacy and transparency duties apply today, and high-risk obligations follow from December 2027.¹⁴

From decision to value

The question of whether to use AI has largely been answered. In McKinsey’s 2025 survey, 88% of organisations used AI regularly in at least one business function.² The question that remains is why so few of them see results. In the 2026 edition, 37% attribute any EBIT impact to AI, about the same as a year earlier, and only around 6% qualify as high performers.¹ BCG’s 2026 survey of 1,330 executives is somewhat more optimistic: it classifies 7.5% of companies as “future-built” and a further 41% as scaling AI and outperforming.³

37%of organisations attribute any EBIT impact to AI¹

~6%are AI high performers¹

60%of AI projects without AI-ready data will be abandoned through 2026, Gartner forecasts⁶

The reasons are not mysterious. Gartner expected at least 30% of generative AI projects to be abandoned after proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs or unclear business value.⁴ It expects more than 40% of agentic AI projects to be cancelled by the end of 2027 for similar reasons.⁵ These are failures of implementation, not of the technology. They are decided by five choices that come after the decision to use AI.

Choice 1: Redesign the workflow, not the task

The strongest single difference between AI high performers and the rest is not the model. Nearly three quarters of high performers report fundamentally redesigning workflows, against one quarter of other respondents.¹ Adding AI to a step in an unchanged process produces a faster step in a process that is still slow. Redesigning the process around what AI does well, and around where people must stay in charge, produces a different result.

In practice, this means answering concrete questions before choosing any technology: where in the process does the AI act, what does it receive, what does it produce, who checks it, and what happens when it is wrong. In the AI workflows I design, every one of these questions has a written answer, including a checklist that the output must pass and a named person who approves it.

Choice 2: The kind of AI

“AI” covers very different technical patterns, with different costs, risks and strengths. Choosing the pattern is the most consequential technical decision of an implementation.

Table 1. Common patterns and when they fit
Pattern	Fits when	Watch for
General model, used as is	Drafting, summarising, general reasoning with low stakes	No access to company knowledge; plausible errors
Retrieval-augmented generation	Answers must rest on company documents or data	Retrieval quality; hallucinations still occur⁸
Fine-tuned model	Narrow, repeated tasks; fixed style or format	Poor at learning new facts;⁷ retraining cost
Several models combined	Different tasks need different strengths or costs	Integration and evaluation effort
Agents with tools	Multi-step work across systems	Excessive permissions; harder to switch providers^10,19

Two findings from research should shape this choice. First, for giving a model access to knowledge, retrieval consistently outperforms unsupervised fine-tuning, both for known and entirely new facts.⁷ Second, retrieval is not a cure: Stanford researchers found commercial legal research tools built on retrieval hallucinating in more than 17% and more than 34% of benchmark queries.⁸ The choice of pattern sets the risk profile; it does not remove the need for checks.

The market reflects this variety. Among 100 enterprise CIOs surveyed by a16z, 37% use five or more models, mainly because different use cases need different strengths. The same study notes that agentic workflows make switching between models harder.¹⁰

Choice 3: Architecture, data and hosting

Data readiness decides more projects than model quality. Gartner forecasts that through 2026, organisations will abandon 60% of AI projects that are not supported by AI-ready data, and found that 63% of organisations lack the right data-management practices for AI or are unsure whether they have them.⁶

Three architecture decisions follow. Where does the data come from, and who maintains it? How does the AI connect to existing systems, and with which permissions? Where does it run? In Europe, the last question has become a legal one as well as a technical one. The EU Data Act has applied since 12 September 2025²⁰ and limits notice periods for switching cloud providers to two months, with switching charges banned from January 2027.²² The European Data Protection Board has stated that an AI model trained on personal data cannot always be treated as anonymous, and that unlawful development can affect the lawfulness of later use.²¹

Cost belongs here too. The price of querying a model at GPT-3.5 level fell more than 280-fold between November 2022 and October 2024.¹¹ Yet savings rarely materialise, because builders move to the newest, most capable models as soon as they appear, according to the investor Menlo Ventures.⁹ A modular architecture, in which the model behind a task can be replaced, protects against both lock-in and cost drift.

Choice 4: Evaluation and human oversight

An AI system without an agreed measure of quality cannot be improved, defended or trusted. The most reliable implementations define, before building, what a good output looks like and test against a set of real cases. Using a strong model to judge outputs can scale this work, with agreement with human judges above 80% in research, but the same research documents systematic biases, for example in favour of longer answers.¹² Automated evaluation supports human judgement; it does not replace it.

Oversight is the second half. For high-risk systems, the EU AI Act requires that people can understand the system’s capabilities and limits, stay aware of automation bias, override or reverse outputs, and stop the system.¹³ These are good design principles for any system whose errors are costly, regulated or not. OWASP’s list of the main security risks of LLM applications, from prompt injection to excessive agency, is a practical checklist for the technical side.¹⁹

Figure 1. A simple oversight chain for outputs that matter. The depth of each gate depends on the cost of an error.

Choice 5: Governance and regulation

Regulation is now a design input with concrete dates. The EU AI Act entered into force on 1 August 2024. Prohibitions and AI literacy duties have applied since 2 February 2025, obligations for general-purpose models since 2 August 2025, and most other rules, including transparency duties, since 2 August 2026, with a grace period until 2 December 2026 for marking AI-generated content in systems already on the market.¹⁴ The Digital Omnibus on AI, Regulation (EU) 2026/1744, in force since 27 July 2026, moved the obligations for high-risk systems listed in Annex III to 2 December 2027 and for those built into regulated products to 2 August 2028.^14,15 It also softened the AI literacy duty: organisations must take measures to support their staff’s AI literacy, rather than guarantee a specific level.¹⁶

Table 2. EU AI Act: key dates as of October 2026
Date	What applies
2 Feb 2025	Prohibited practices; AI literacy
2 Aug 2025	Obligations for general-purpose AI models
2 Aug 2026	Most remaining rules, including transparency
2 Dec 2026	New prohibitions (sexual deepfakes, child sexual abuse material); end of the grace period for marking AI-generated content
2 Dec 2027	High-risk systems listed in Annex III
2 Aug 2028	High-risk AI in products covered by Annex I

Beyond legal duties, voluntary frameworks give governance a structure: the NIST AI Risk Management Framework with its generative AI profile,¹⁷ and ISO/IEC 42001 for AI management systems.¹⁸ For a small organisation, the useful core is simple: an inventory of AI uses, a risk rating for each, a named owner, and a record of how each system is evaluated and overseen. This article gives general information, not legal advice.

Why context decides

It is tempting to assume that the most capable solution is the best one: the largest model, the most autonomous agent, the most integrated platform. In practice, the right solution depends on the context in which it runs. Two organisations with the same use case can need different answers.

Table 3. The same use case, different contexts, different right answers
Context	Usually the better choice
High cost of error, regulated domain	Retrieval with strict sources, narrow scope, approval for every output
Low cost of error, high volume	General model with automated checks and sampling
Sensitive or personal data	Hosting and providers chosen for data protection first; minimal data in prompts
Small team, little AI experience	Fewer moving parts; managed services; one well-evaluated workflow before the next
Data scattered or poorly maintained	Data work first; AI on the parts that are ready
Fast-changing processes	Modular design with replaceable models; avoid deep fine-tuning

The same logic applies to autonomy. An agent that acts across several systems can be the right choice for a team with mature permissions, logging and review. For a team without them, a simpler assistant that prepares work for a person to complete often delivers more value with far less risk.

The organisations that succeed with AI are rarely the ones with the most sophisticated technology. They are the ones whose choices fit their workflow, their data, their people and their obligations, and who revisit those choices as each of them changes.

Frequently asked questions

Should we fine-tune a model or use retrieval?

For giving a model access to company knowledge, research finds that retrieval consistently outperforms unsupervised fine-tuning. Fine-tuning is better suited to style, format or narrow tasks. Many systems combine a general model with retrieval.

Does retrieval-augmented generation eliminate hallucinations?

No. It reduces them, but a Stanford study of commercial legal research tools built on retrieval still found hallucination rates above 17% and above 34%. Evaluation and source checks remain necessary.

Should we use one AI provider or several?

Many enterprises use several models for different tasks. A modular architecture keeps switching possible, while agentic workflows tend to make switching harder.

When do the EU AI Act duties apply?

The act entered into force on 1 August 2024. Prohibitions and AI literacy have applied since February 2025, general-purpose model duties since August 2025, and most other rules since August 2026. Following the Digital Omnibus, high-risk duties for Annex III systems apply from 2 December 2027.

How do we know whether an AI system works well enough?

Define what good looks like before building, create a test set from real cases, measure regularly, and keep a person responsible for outputs where errors are costly.