How do you judge short form video agency reviews in England?

What to take away

The ICO's direct marketing guidance, updated in 2024, is the reference point for the lawful basis behind any data-driven short-form video campaign.
Most published agency reviews describe creative output and account management, not measurement, data handling or contract terms.
A review is a claim, not evidence. Treat it as a lead, then verify separately against your own brief.
Under the Data Protection Act 2018, the agency processing your audience data carries duties you remain accountable for.

What these reviews actually cover

This piece reviews how agency reviews are written, not how any named agency performs. No agency was briefed, tested or contacted. Nothing here is hands-on testing, and no vendor supplied material for it.

Most reviews found online assess three things: creative quality, speed of response and value for money. Those are editorial judgements made by whoever published them. They are useful signals, but they are opinions based on somebody else's brief.

What reviews rarely assess is harder to judge from the outside. That includes pixel and tag governance, retention of audience data, and who owns the raw footage. It also includes what happens to your accounts when the contract ends. Ask about those directly rather than searching for a review.

Vendor claims versus independent evidence

Agency case studies are vendor claims. An award shortlist is a third-party signal, though it usually reflects one campaign rather than a repeatable process. A named client reference you can telephone is stronger than either.

Evidence Strength Compared

Vendor claim

Source: Agency case study
Scope: Self-reported
Verification: None
Strength: Weakest

Third-party signal

Source: Award shortlist
Scope: One campaign
Verification: Partial
Strength: Medium

Independent evidence

Source: Written method
Scope: Repeatable process
Verification: Dated report
Strength: Strongest

Independent evidence looks different. It includes a written method, a dated performance report and a clear statement of what the agency will not do. If a review repeats a claim about reach or conversion, check whether the reviewer saw the underlying dashboard.

When a review praises targeting, ask what lawful basis the campaign used. The ICO's guidance on choosing your lawful basis for direct marketing explains why consent and legitimate interests lead to different obligations.

Data handling questions to ask

Short-form video work often involves audience lists, lookalike audiences and retargeting pixels. That means personal data changes hands. The Data Protection Act 2018 sets the framework your agency must work within.

Data Handling Questions

Who is controller and who is processor?
Where is audience data stored?
How long is data kept?
What happens to custom audiences at exit?
Is there a written processing agreement?
Can retention schedule be stated plainly?

Ask who is controller and who is processor, where data is stored, and how long it is kept. Ask what happens to custom audiences when the engagement ends. A review will not answer these, but a written data processing agreement will.

If an agency cannot describe its retention schedule in plain language, that is a finding in itself. Record it in your scoring notes.

Creative standards to expect

Reviews tend to reward work that looks polished. Buyers should also check whether the creative follows recognised advertising standards. The IAB UK introduction to creative best practice for digital ads sets out expectations that apply to vertical video as much as to display.

Ask to see the first three seconds of five recent assets. Then ask what the brief was and how performance was judged. A review that only shows finished films tells you little about process.

A scoring method you can defend

Score each agency on the same weighted criteria, then keep the evidence beside the score. Our guide to short form video reviews and comparison methods sets out a weighting model you can adapt.

Separate what you verified from what you were told. A reference call is verification. A pitch deck is not. Note the date of every figure you record.

Use a checklist so nothing is skipped when three agencies are compared at once. The short form video selection checklist covers the documents to request before scoring begins.

Decision table

Choose

First campaign, small budget: Agency with a written method and named references
Regulated sector, audience data involved: Agency that documents lawful basis and retention
In-house team needs capacity: Agency offering training and handover
Fast turnaround on vertical video: Agency with a dated production schedule
Long-term brand building: Agency with quarterly measurement reviews

Avoid

First campaign, small budget: Agencies whose only proof is awards
Regulated sector, audience data involved: Agencies that cannot name a processor
In-house team needs capacity: Agencies that keep account ownership
Fast turnaround on vertical video: Agencies promising same-day edits
Long-term brand building: Agencies reporting only views

Common questions

Are online agency reviews reliable?

They are opinions, usually based on one client relationship. Treat them as leads to verify, not as evidence of fit for your brief.

What should I check that reviews miss?

Data processing terms, asset ownership, retention periods and exit arrangements. Ask for these in writing before you score any agency.

Do I need a lawyer to review the contract?

For campaigns handling personal data, legal review of the processing terms is sensible. The ICO guidance above is a starting point, not legal advice.

How many agencies should I score?

Three is workable for most teams. Score all of them against the same weighted criteria so the comparison stands up internally.