Project Management

Why Most Computer Vision Projects
Fail in Practice

By Dr. Ali Khan  |  Occulins Lab
October 5,2026


Most computer vision projects do not fail because the technology does not work. The technology works. Detection models, segmentation models, classification models, the tools are mature, well-documented, and capable of solving genuinely difficult problems when applied correctly.

Most CV projects fail for reasons that have nothing to do with the model. They fail because success was never defined precisely before starting. They fail because the data problem was underestimated by a factor of three. They fail because the people commissioning the project expected a plug-and-play solution and the people building it did not correct that expectation early enough. They fail because nobody planned for what happens six months after deployment when the model encounters conditions it was not trained on.

This post covers five failure patterns I have seen repeatedly. None of them are technical failures. All of them are preventable. And every one of them could have been avoided with a direct conversation before the project started.

Figure 1
Where CV projects actually fail
Where CV projects actually fail, mapped to project stages. Technical failures (model performance, architecture choices) account for a minority of project failures. The majority originate in the definition, data, and expectation stages before a single model is trained.

1
No Defined Success Criteria Before Starting

This is the most common failure mode and the one that causes the most expensive outcomes. A project begins with a goal stated in qualitative terms, "detect defects on the production line," "identify lesions in dermoscopic images," "count vehicles in aerial footage." Everyone agrees the goal sounds reasonable. No one defines what success looks like in measurable terms before work begins.

The consequence is that the project has no finish line. Work continues until the budget runs out, or until someone decides the model is "good enough," or until the team and the client have sufficiently different ideas of what good enough means that the relationship breaks down. None of these are good outcomes.

What Undefined Success Looks Like

The technical team trains a model that achieves 78% mAP. Is that success? It depends entirely on questions that should have been answered before the project started: what is the minimum acceptable mAP for the application, what is the acceptable false positive rate, what is the acceptable false negative rate, and are these requirements the same for all classes or different by class.

In a defect detection application, a false negative, a defective product that passes inspection, has a very different cost than a false positive, a good product that is rejected. If those costs were not quantified before training started, the evaluation at the end is meaningless. 78% mAP might be excellent for this application or completely unacceptable, and without defined criteria, there is no way to know which.

How this failure pattern appears The project produces a working model. The client reviews it and says it is not quite right. The team makes adjustments. The client reviews again. This cycle continues without convergence because there is no agreed definition of what "right" means. Eventually someone runs out of patience or budget.
What prevents it Before any work begins, define success in writing: minimum acceptable accuracy per class, maximum acceptable false positive rate, maximum acceptable false negative rate, target latency, target hardware, and the evaluation methodology that will be used to measure all of the above. Both parties sign off. This document is the finish line. Everything else is navigation toward it.
The single most valuable conversation I have had in CV projects is the one where I ask the client "what number would make you happy and what number would make you refuse to deploy this?" Getting a specific answer to both questions before starting, not during or not at the end, is what makes it possible to build something that meets the requirement rather than something that is iteratively adjusted until everyone is exhausted.
2
Underestimating the Data Problem

The most consistent underestimation in CV projects is the cost, time, and complexity of producing training data. People who have not run a real annotation project tend to think of data collection as a preliminary step — something you do before the real work begins. In practice, data is the real work. The model training is relatively straightforward once the data is right. Getting the data right is where most of the time goes.

What Annotation Actually Costs

Consider a detection project requiring 5,000 annotated images with bounding boxes. An experienced annotator working on a moderate-complexity task can label approximately 50 to 100 images per hour. At 100 images per hour, 5,000 images requires 50 hours of annotation time, a full working week for one person. That estimate assumes: the annotation tool is set up, the class definitions are clear and unambiguous, quality review is not included, and the annotator encounters no difficult edge cases.

In practice: annotation tools require setup and training time. Class definitions require iteration because edge cases that were not anticipated emerge during annotation. Quality review,comparing annotator work against a standard and correcting errors, typically adds 20 to 40% to the total time. Difficult edge cases require expert review that takes longer than standard annotation. A realistic estimate for 5,000 annotated detection images, including setup, annotation, quality review, and iteration on class definitions, is 100 to 200 hours that is two to four times the naive estimate.

For segmentation tasks, where masks must be drawn at pixel level rather than bounding boxes, annotation time is typically three to five times higher than detection annotation for the same number of images.

Figure 2
Expected vs actual project timeline and cost breakdown
Expected vs actual project timeline and cost breakdown across five CV project failure patterns. The data stage consistently takes two to three times longer than initially estimated. Model development and evaluation are more predictable. Post-deployment work including monitoring, maintenance, retraining, is almost never budgeted for at project start.

The Hidden Data Problems

Beyond annotation cost, several data problems consistently appear mid-project that were not anticipated at the start.

Class definition ambiguity. A class that seemed obvious during planning becomes ambiguous during annotation. "Defect" in industrial inspection encompasses dozens of visually distinct defect types. "Lesion" in dermoscopy includes objects at the boundary of what experts would call a lesion. Resolving these ambiguities requires expert input, annotation protocol revision, and re-annotation of images that were already labelled under the previous definition.

Data acquisition gaps. The training data does not cover conditions that appear at deployment. The model was trained on daytime images and deployed in a facility that operates at night. The model was trained on a specific camera model and deployed with a different camera. The model was trained on a clean industrial line and deployed in a facility with more background clutter. Each gap requires new data collection and annotation.

Label quality degradation. Annotation quality decreases when annotators are rushed, fatigued, or poorly briefed. A dataset with 10% annotation errors such that boxes drawn too loosely, ambiguous cases labelled inconsistently, objects missed, will produce a model that cannot be trusted regardless of how well it is trained.

How this failure pattern appears The project starts with an optimistic data collection plan. Two months in, the team has half the data they expected and the annotation quality is inconsistent. The model training is delayed while data issues are resolved. The budget allocated for model development is consumed by data remediation. The deployment date slips.
What prevents it Treat data as the primary project deliverable, not a prerequisite. Budget annotation time at three times the naive estimate. Run a small annotation pilot, 100 to 200 images, before full annotation begins, to surface class definition ambiguities and calibrate annotator speed. Build quality review into the annotation process from the start, not as an afterthought.
3
Treating Computer Vision as Plug-and-Play

The accessibility of modern CV frameworks such as Ultralytics, Hugging Face, TensorFlow Hub, has created a reasonable but incorrect impression that CV is a commodity. You download a pretrained model, fine-tune it on your data, and deploy it. The impression comes from the fact that this works remarkably well for standard problems with standard data in standard deployment conditions.

Most real business problems are not standard. The camera angle is unusual. The objects are small and densely packed. The deployment hardware is constrained. The lighting conditions vary. The required accuracy is higher than what off-the-shelf fine-tuning achieves. When these non-standard conditions are present and the plug-and-play assumption is not corrected early, the project encounters a gap between what was expected and what is achievable, and that gap is discovered late, when it is expensive to close.

What Plug-and-Play Actually Provides

Pretrained models and fine-tuning are genuinely powerful starting points. They reduce the time to a working baseline from months to days. For standard conditions, reasonable image quality, reasonable object sizes, standard detection or classification tasks, fine-tuning often produces acceptable results with modest data and effort.

What plug-and-play does not provide: guaranteed accuracy on non-standard data, reliable performance on rare classes with few training examples, appropriate behaviour in edge cases that were not in the training distribution, or performance guarantees for deployment conditions that differ from the pretraining conditions.

The gap between "a working baseline" and "a reliable production model" is where most of the real work lives. A baseline that achieves 65% mAP in two days of fine-tuning may require two months of data engineering, architecture adjustment, and evaluation iteration to reach 85% mAP, if 85% is even achievable given the data and conditions of the problem.

The honest expectation A good CV practitioner can get you a working baseline quickly. Getting from that baseline to a model that meets your specific accuracy, latency, and reliability requirements on your specific data and deployment hardware is the work that takes time, expertise, and iteration. The baseline is the beginning of the project, not the end of it.
How this failure pattern appears A demo is produced quickly and looks impressive. Stakeholders conclude the project is nearly done. The remaining work — closing the gap between demo performance and production requirements — takes three times as long as the demo. Stakeholder expectations do not adjust proportionally. The gap between expectations and progress grows until someone is disappointed.
What prevents it Present the baseline as a baseline explicitly, not as a near-complete solution. Communicate clearly that the distance between a working demo and a production-ready model is where most of the project effort lives. Set expectations about the iteration process before the demo is seen, not after.
4
Unrealistic Timeline and Budget Expectations

CV project timelines are consistently underestimated for three compounding reasons: the data work takes longer than planned, the gap between baseline and production quality requires more iteration than planned, and the evaluation and integration work at the end of the project is not planned for at all.

A Realistic Project Timeline

Stage Naive estimate Realistic estimate Why the gap exists
Data collection and annotation 2 weeks 4–8 weeks Class ambiguity, quality issues, coverage gaps
Baseline model training 1 week 1–2 weeks Relatively predictable
Iteration to production quality 1 week 4–8 weeks Multiple training cycles, data augmentation, evaluation
Evaluation and validation 2–3 days 1–2 weeks Per-class analysis, qualitative review, edge case testing
Integration and deployment 1 week 2–4 weeks Pre-processing pipeline, format conversion, system integration
Total 5–6 weeks 12–24 weeks 2–4× the naive estimate

The budget gap follows the same pattern. The naive estimate budgets for model development time. The realistic estimate includes data annotation cost, multiple iteration cycles, evaluation infrastructure, and integration work. The difference is typically a factor of two to three.

The most useful thing I can tell someone planning a CV project is to take their timeline estimate, double it, and then ask whether they still want to proceed. If doubling the timeline makes the project unviable, the project was not going to succeed at the original timeline either, it was going to fail under time pressure instead of succeeding on a realistic schedule.
How this failure pattern appears The project is scoped and budgeted based on the naive timeline. Halfway through, the team is behind schedule and the budget is nearly consumed. Quality is sacrificed to meet the deadline. The model is deployed before it is ready. Post-deployment problems require emergency remediation that costs more than the original project.
What prevents it Use the realistic timeline estimates above as a starting point. Build in explicit buffer for data issues, they always occur. Separate the budget into phases with defined deliverables at each phase boundary, so budget consumption is visible before it becomes a crisis.
5
No Plan for After Deployment

A deployed CV model is not a finished product. It is a system component that requires maintenance, monitoring, and periodic updating, like any other software component in production, but with the additional complexity that its performance depends on the distribution of real-world inputs, which changes over time in ways that are difficult to predict.

Most CV projects plan for development and deployment. Almost none plan for what happens after deployment. This is the failure mode that makes itself felt the slowest and is the most expensive to remediate once it appears.

What Happens After Deployment Without a Plan

In the first weeks after deployment, the model encounters the exact conditions it was trained on and performs as expected. Over the following months, conditions drift. A new product variant appears on the production line that was not in the training data. Seasonal changes affect lighting conditions in an outdoor deployment. The camera is replaced with a different model. A new demographic of patients starts presenting at a medical facility.

Without monitoring, none of these drifts are detected until someone notices that the model's output looks wrong — which typically happens when a problem has been accumulating for weeks or months. Without a retraining process, fixing the drift requires starting a new data collection and training project from scratch. Without a versioning strategy, deploying the updated model risks introducing regressions in cases that previously worked.

How this failure pattern appears The model is deployed and performs well initially. Six months later, a stakeholder notices that detection accuracy seems lower than it used to be. Nobody is sure when it started degrading or why. There is no monitoring data to diagnose the issue. Fixing it requires essentially restarting the project with new data.
What prevents it Design the post-deployment plan before deployment. Define: what metrics will be monitored, how often, and by whom. What threshold triggers an investigation. What triggers a retraining cycle. How new training data will be collected from deployment failures. How updated models will be validated before switching production traffic. None of this is technically complex — but it requires deliberate planning and a small ongoing operational budget that most projects do not include.

What Successful Projects Have in Common

The pattern that works

Projects that succeed share five characteristics that are the direct inverse of the five failure modes above.

Success is defined in writing before work starts. Minimum accuracy thresholds, acceptable error rates per class, latency requirements, and the evaluation methodology are agreed and documented at the beginning, not discovered at the end.

Data is treated as a primary deliverable. Annotation cost is estimated realistically, a pilot annotation run surfaces ambiguities before full annotation begins, and quality review is built into the process rather than appended to it.

The gap between baseline and production is communicated clearly. Stakeholders understand before the first demo that a working baseline is the beginning of the iteration process, not the end of the project. Timeline and budget reflect this reality.

Timeline estimates are realistic. The project plan uses the realistic estimates in the table above, not the naive ones. Buffer is built in for data issues. Phase boundaries with defined deliverables provide visibility into progress before problems become crises.

Post-deployment operations are planned before deployment. Monitoring, retraining triggers, and model versioning are designed into the system before it goes live — not added reactively when something goes wrong.


Figure 3
Five questions every CV project must answer before starting
The questions every CV project should answer before work begins. Each question corresponds to one of the five failure modes. A project that cannot answer all five clearly before starting is a project that has not yet done the work required to succeed.

The Honest Summary

Computer vision is a mature and capable technology. The failure rate for CV projects is not primarily a technology problem, it is a planning and expectation problem. The projects that fail do not fail because the models do not work. They fail because the surrounding conditions such as defined success, realistic data plans, accurate timelines, post-deployment operations, were not put in place before the technical work began.

A CV practitioner who tells you these things before you commission a project is more valuable than one who tells you what you want to hear. The honest conversation at the start is what makes the difference between a project that delivers and one that consumes budget and produces disappointment.

If reading this post makes a CV project sound harder and more expensive than you expected, that is the correct reaction. It is harder and more expensive than most people expect, and knowing that before you start is the most valuable thing this post can give you.

Selected implementations, supporting utilities, and companion resources related to this article are available through the Blog 13 Companion Resources page .

Companion Resources Included

  • Realistic project timeline reference
  • Five questions before starting checklist
  • Failure prevention checklist

Working on a CV project and want to get the planning right before work begins?

The five questions in Figure 3 are a useful starting framework. If you want to work through them for your specific application — defining success criteria, estimating data requirements, and building a realistic project plan — feel free to reach out through Occulins Contact

Tags: Computer Vision Project Management Data Annotation Deployment Expectations Deep Learning Production ML Object Detection Semantic Segmentation