In 2026 nearly every product launch comes with an "AI-powered" badge. Most of the time that badge is marketing and nothing more; the AI isn't really adding anything a user would miss.
Put it in the right spot, though, and it genuinely changes the product. Whenever we add an AI feature to one of our own products at AppDuce, the first question is always the same: does this actually help the person using it?
What follows isn't theory. It's how you go about bolting AI onto an app that already exists.
Questions to ask before you add AI
First, do users actually want it? Adding AI feels cool, but that's not a reason. If people already know what they came to do, a "smart recommendation" sitting on top of that just gets in the way.
Second, can you do the job by hand first? Before you write a line of AI code, try doing the same work manually. If you're dreaming up "smart categorization," go categorize 100 records yourself. You'll learn fast how good the model actually needs to be, and sometimes that the rules in your head are enough.
Third, what happens when it's wrong, because it will be? A recommendation engine that suggests the wrong movie is something people shrug off. A medical app that hands out wrong information is a different category of problem. Be honest about how much a mistake costs you.
Four practical ways to add AI
1. LLM APIs (GPT, Claude, Gemini)
This is the quickest and by far the most common route: fire a request at a language model over its API and use what comes back.
It fits a handful of jobs well:
- Summarizing long text
- Chatbots and customer support
- Generating content like product descriptions or email drafts
- Pulling structured data out of messy text
- Search that understands plain language
Wiring it up looks roughly like this:
- Get an API key from OpenAI, Anthropic, or Google
- Build a backend endpoint that makes the call
- Write a prompt that encodes your business logic
- Handle the response and show it to the user
A word on cost, since it surprises people. Models like GPT-4o or Claude Sonnet bill per token. A simple chatbot might run $50-500 a month, and heavy usage climbs into the thousands. Swapping in a lighter model like GPT-4o-mini or Claude Haiku for the simple tasks cuts that bill down a lot.
2. Image and audio processing
For images, you're usually after one of a few things: identifying a product from a photo, reading text off a document with OCR, or visual search. Google Vision, OpenAI Vision, or a model you train yourself all cover this.
For audio, it's voice commands, speech-to-text, and transcription. Whisper from OpenAI is open source and very good, and it's where I'd start.
One practical note: both of these almost always run on the server. On-device options exist, like Core ML and TensorFlow Lite, but you'll hit their limits quickly.
3. Recommendation systems
Here the goal is suggesting things based on how someone behaves, and it comes in three rough tiers.
The simple version is rule-based: "people who bought X also bought Y," which is just a database query and doesn't need AI at all. The middle tier uses similarity, turning product features into vectors and finding near matches with something like pgvector in Postgres or a dedicated vector database like Pinecone. The advanced tier trains a model on your behavior data, which wants a large dataset and is usually overkill for a small app.
4. Automatic classification and analysis
This is about sorting messy, unstructured data on its own. Common uses:
- Bucketing customer feedback into complaint, suggestion, or praise
- Ranking support tickets by urgency
- Tagging content automatically
- Reading sentiment as positive, negative, or neutral
A big LLM handles all of these, but you don't always need one. A lighter, cheaper option like a fine-tuned BERT or a small custom classifier often does the same job for less.
Technical architecture
How you wire the AI in matters as much as which model you pick. The basic flow looks like this:
User → Frontend → Backend API → AI Service → Response
↓
Cache Layer
A few rules we don't break:
- Make the AI calls from the backend, never the frontend. That keeps your API keys out of reach and your costs under your control.
- Cache aggressively. There's no reason to pay for the same answer to the same question twice, and a Redis or plain database cache pays for itself fast.
- Set a timeout and a fallback. When the AI service is slow or down, the user should see a "couldn't get a response" message, not a frozen blank screen.
- Rate limit per user. One person firing off 100 requests an hour is how costs blow up and how abuse creeps in.
- Stream long responses. Sending the answer word by word instead of making people wait for the whole thing is a huge difference in how fast it feels.
Prompt engineering basics
What you get out of a model tracks closely with the prompt you put in. A few things carry most of the weight.
The system prompt sets the model's role, something like: "You are a customer support assistant. Only answer questions about our product. If you're not sure, say you don't know." Beyond that, give it the context it needs to answer, whether that's product details, the user's history, or the page they're on. Be explicit about the output format you want too, with instructions like "reply in JSON," "summarize in three bullets," or "keep it under 150 words." And put guardrails in place so it won't wander off topic or produce something harmful.
Managing cost
AI features cost real money. Here's where the savings actually come from:
| Strategy | Savings | Implementation |
|---|---|---|
| Model selection | High | Use smaller models for simple tasks |
| Cache | Medium-high | Cache repeated requests |
| Token limits | Medium | Set max input/output tokens |
| Rate limiting | Low-medium | Per-user request limits |
| Batch processing | Medium | Batch process when real-time isn't needed |
A real example: ShortsByAuto
Take our own product, ShortsByAuto, which turns long videos into short clips. The AI shows up in a few specific places: it analyzes the footage to find the moments most likely to land, it generates subtitles, and it suggests titles and descriptions for each clip. But we check its output at every one of those steps, and the person, not the model, makes the final call. The AI is a tool here, not the one in charge.
When not to add AI
Sometimes the right move is to skip AI entirely, and it's worth naming when.
If you don't have data, personalization has nothing to work with; a recommendation engine for your first 100 users makes no sense. If a plain rule does the job, use the rule, because "free shipping over $50" doesn't need a model. If you can't tolerate latency, remember that an AI call takes anywhere from half a second to three seconds, which is unacceptable in places that need to feel instant. And if the budget is tight, know that AI cost tends to grow with usage, so try the free tiers before you commit to anything.
Where it earns its place
Does every app need AI? No. But dropped into the right spot and built with some care, it makes the experience noticeably better.
Keep the first step small. Add one feature backed by an LLM API and watch how people respond. If it earns its keep, build on it. If it doesn't, pull it back out. And if you want to point AI at your own development process while you're at it, vibe coding is worth a read.
We use AI where it earns its place, and that place is our own products. We build in the open, so you can follow along and watch it happen.