DeepSeek V4.1 Flash / Existing primaries
Application fit.
Reliability.
Same inputs. The same application processing. Quality is yours to judge.
PAIRED RUN COMPLETE · Routing unchanged
A schema pass alone is not an application pass. Both model outputs go through the same checked application path. Some input histories are incomplete, and several features have fewer than 5–6 independent historical runs. Geo lookup, persistence, rendering and publishing are outside this read-only evaluation.
Observed compatibility and reliability
Counts are per paired case. “Application accepted” includes valid clarifications; completed operations are reported inside each service. No quality or rollout grades.
| Service | Pairs | Existing app accepted | DeepSeek app accepted | Existing first valid | DeepSeek first valid | Existing median | DeepSeek median |
|---|---|---|---|---|---|---|---|
| Business text analysis | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 2.0s | 20.8s |
| Website visual identity | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 2.9s | 18.1s |
| Scraped-photo selection | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 2.3s | 29.7s |
| Business photo analysis | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 4.4s | 59.5s |
| Fashion classification | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 2.7s | 5.7s |
| Quick photo triage | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 2.5s | 10.4s |
| Brand intelligence | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 5.8s | 70.1s |
| Brand visual system | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 5.4s | 69.2s |
| Monthly calendar strategy | 6 | 6/6 | 5/6 | 6/6 | 1/6 | 21.2s | 292.1s |
| Calendar variants | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 6.2s | 21.1s |
| Caption shortening | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 3.1s | 17.7s |
| Copy after photo changes | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 3.3s | 19.7s |
| Calendar refresh | 5 | 5/5 | 4/5 | 5/5 | 3/5 | 12.2s | 105.3s |
| Batch video ideas | 6 | 6/6 | 5/6 | 6/6 | 1/6 | 21.5s | 258.7s |
| Onboarding video idea | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 4.4s | 17.5s |
| Trending video ideas | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 8.3s | 50.2s |
| Storyboard compilation | 6 | 4/6 | 5/6 | 6/6 | 1/6 | 10.4s | 335.1s |
| Storyboard style patch | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 3.9s | 12.1s |
| Image prompt enhancement | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 9.2s | 20.6s |
| Product pack briefs | 4 | 4/4 | 4/4 | 4/4 | 4/4 | 6.6s | 112.4s |
| Product pack refinement | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 4.3s | 15.7s |
| Photoshoot ideas | 6 | 6/6 | 1/6 | 6/6 | 1/6 | 15.8s | 120.0s |
| Attachment descriptions | 6 | 6/6 | 2/6 | 6/6 | 2/6 | 2.7s | 12.0s |
| Carousel planning | 6 | 6/6 | 5/6 | 6/6 | 4/6 | 3.0s | 9.3s |
| Native website generation | 6 | 6/6 | 2/6 | 6/6 | 0/6 | 35.8s | 1071.4s |
| Native website editing | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 4.3s | 5.6s |
| Website builder prompt | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 4.5s | 10.1s |
| Website theme suggestions | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 2.3s | 18.0s |
| Click-to-WhatsApp ad copy | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 4.5s | 21.6s |
| Engagement ad copy | 6 | 6/6 | 6/6 | 6/6 | 5/6 | 3.7s | 31.1s |
| Google post polishing | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 2.7s | 3.8s |
| Google post writing | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 4.6s | 9.9s |
| AI video editing director | 6 | 6/6 | 6/6 | 6/6 | 5/6 | 4.8s | 14.3s |
| Avatar gender labels | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 1.8s | 6.1s |
| Garment analysis | 6 | 6/6 | 6/6 | 6/6 | 5/6 | 3.6s | 20.8s |
| Garment prompt writing | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 5.1s | 42.0s |
| Product analysis | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 4.6s | 9.7s |
| Product prompt writing | 6 | 6/6 | 6/6 | 6/6 | 5/6 | 7.2s | 22.6s |
| Avatar promotional script | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 3.1s | 15.8s |
| Voice script processing | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 3.3s | 20.3s |
| Sora prompt enhancement | 6 | 6/6 | 6/6 | 6/6 | 6/6 | 10.0s | 12.9s |
What the results establish
The comparison uses a fresh existing-primary response and a fresh DeepSeek response for every case. Saved historical outputs are not used as the baseline or as ground truth.
Provider request schemas follow the source transport where it differs from the logical contract. Avatar/voice use the application’s closed-object schema; Gemini storyboards use the installed Google AI SDK OpenAPI converter. Earlier malformed harness requests are archived and excluded, and the affected pairs are rerun. The logical output contract and application validator are unchanged.
Both sides receive the same frozen initial system/user text, conversation messages, logical contract, ordered image bytes and total output-token ceiling. Retry feedback describes each model’s own preceding validation errors under the same retry policy. The same deterministic application processing is then applied. Root arrays use the same transport wrapper and are unwrapped before application processing. Raw outputs remain separately visible. Website requests use the source provider-schema sanitizer and its complexity exceptions. Generation uses the source acceptLooseOutput gate; raw schema differences are diagnostics, while the document normalizer and validator determine application acceptance.
Current primary models and verified per-service output limits are retained. Thinking uses the current primary setting and the closest available DeepSeek effort (disabled, low or high). Gemini budgets/defaults and DeepSeek efforts are not equivalent amounts of computation; requested settings are exposed per service.
Reliability and latency describe the tested model, reasoning settings and provider routes. They do not establish how other DeepSeek effort levels or provider routes would behave. Calls ran in bounded concurrent evaluation workers; this was not a controlled throughput benchmark.
Where source timeouts are verified, they are applied to both arms. Else the shared evaluation ceiling is labelled explicitly. Elapsed time includes request retries/backoff but excludes cached image preparation. Provider-internal retries and failover inside OpenRouter are not independently observable.
First-call validity, validity after retries, application acceptance and operation completion are separate metrics. A valid clarification or no-op is not automatically a completed edit. Application validators run after the provider/JSON retry policy; this is not a replay of every production repair loop, workflow timeout, renderer, database write or publication API.
All image inputs use identical cached bytes; HEIC conversions from the previous dataset are reused for both arms. Historical prompts were reconstructed where exact payloads were not stored. Storyboard prompts were rebuilt from saved server-owned brief/assets/duration and the correct format field; unavailable historical idea context is not invented.
The 243-case snapshot contains 159 historical reconstructions and 84 fresh requests on staging inputs. Product packs have four product calls from two plan runs. Refresh has five batches from one run. A read-only staging count on 14 September confirmed there are still only two saved pack plans and one saved refresh run. Refinement, caption shortening and reference-copy edits reuse inputs or representative instructions. These do not satisfy 5–6 independent historical runs per feature; each case retains its evidence limitation.
Application outputs use source parsers, normalizers and pure assembly functions, with provider and persistence boundaries injected in memory. Ad recommendations are checked with external geo lookup disabled, so ad launch compatibility remains partially verified. Text outputs are not sent to image/video/TTS providers or public posting APIs.
Source bytes used by application replay are retained privately. A concurrent edit to the refresh source file was audited: all five frozen system prompts, user prompts and schemas still match that source. Both arms were checked against the same recorded application source; this work did not change application code.
Offline fault-injection tests verify that a candidate gate can delegate to the unchanged Gemini→GPT and GPT→Gemini wrappers in order, and that exhausting providers fails. These are harness tests, not measured live outage recovery or an implemented production rollout.
Operator-interrupted and superseded configuration attempts are retained in the private audit files and excluded from model reliability rates. Returned OpenRouter usage gives only known candidate charges; unreported interrupted calls and native-provider costs are not included.
No quality scores or replacement verdicts are assigned. Six cases are a compatibility smoke sample and cannot establish a production reliability guarantee. Model routing remains unchanged pending report approval.
Provider protocol references: OpenRouter reasoning · OpenAI structured outputs.