Start with the workflows that create risk or repeated work in your operation. A long feature list does not show whether a system can preserve lot identity through production, calculate a variable-weight order correctly or recover when a delivery does not go to plan. Your evaluation should.
The evaluation scorecard
Ask every vendor to run the same six tests.
Give vendors anonymised but representative data and score what they can demonstrate today separately from configuration, integration work and roadmap promises.
Traceability
Scenario to run
Trace a dispatched item back through any production step to receiving, then identify every affected customer from a source lot.
Evidence to look for
Live result using representative lots, exceptions and exports, not only a slide or pre-arranged happy path.
Catch weight
Scenario to run
Order a variable-weight line, capture an out-of-tolerance actual weight and show the resulting commercial value.
Evidence to look for
Target, actual, tolerance, user/device provenance, price calculation and exception handling on one transaction.
Order to invoice
Scenario to run
Move a mixed order through allocation, shortage, picking, dispatch, POD and invoice-ready processing.
Evidence to look for
One connected record plus an explicit explanation of approvals, integration boundaries and failure recovery.
Quality records
Scenario to run
Show how a failed intake or production check is contained, reviewed and audited without editing history away.
Evidence to look for
Roles, timestamps, status changes, attachments or notes and the practical hold/release workflow.
Security
Scenario to run
Ask how one customer's data is separated from another's and which actions produce usable audit records.
Evidence to look for
Architecture and control explanations tied to implemented behaviour, supported by appropriate assurance where available.
Implementation
Scenario to run
Map the first go-live phase, migration rehearsal, training, acceptance criteria, rollback and ownership after launch.
Evidence to look for
Named responsibilities, realistic dependencies, representative test cases and a written boundary around custom work.
Reusable demo worksheet
Record what happened, not how polished the demo felt.
Set your priorities and expected results before the meeting. Ask every vendor to run the same scenarios, then record the evidence level separately from the work still needed to deliver it.
1. Define the pass
Write the result, data and exception that matter before seeing the product.
2. Use safe data
Use anonymised but representative products, lots, weights and roles.
3. Save evidence
Record the screen, export or audit entry and the version or date shown.
4. Name the dependency
Separate available behaviour from configuration, integration, custom work and roadmap.
Evidence scale
The number is a prompt for consistent comparison, not a universal procurement formula. Agree what counts as a pass for each critical scenario before the demonstrations begin.
- 0Not shown
- No usable evidence was provided.
- 1Talk-through only
- The vendor described the result or used slides.
- 2Clean path shown
- Representative data worked, but the awkward exception was not tested.
- 3Operational proof
- Representative data, the exception, recovery and saved evidence were shown.
Vendor comparison sheet
Mark each area as must pass, important or useful. Add the 0–3 evidence level, then write one delivery status: available now, configuration, integration, custom build, roadmap or unknown.
Print or copy one sheet per vendor
Swipe the sheet horizontally to see the evidence and delivery columns.
| Scenario | Priority and pass result | Exception to test | Evidence and level 0–3 | Delivery status, gap and owner |
|---|---|---|---|---|
| Traceability | Priority: ______ Pass: __________________ | Lot split, hold or incomplete record Your detail: ____________ | Level: __ / 3 Evidence: ______________ | Status: _________________ Gap / owner: ____________ |
| Catch weight | Priority: ______ Pass: __________________ | Out-of-tolerance actual weight Your detail: ____________ | Level: __ / 3 Evidence: ______________ | Status: _________________ Gap / owner: ____________ |
| Order to invoice | Priority: ______ Pass: __________________ | Shortage, failed delivery or credit Your detail: ____________ | Level: __ / 3 Evidence: ______________ | Status: _________________ Gap / owner: ____________ |
| Quality records | Priority: ______ Pass: __________________ | Failed check and controlled release Your detail: ____________ | Level: __ / 3 Evidence: ______________ | Status: _________________ Gap / owner: ____________ |
| Security | Priority: ______ Pass: __________________ | Role boundary, audit evidence or recovery Your detail: ____________ | Level: __ / 3 Evidence: ______________ | Status: _________________ Gap / owner: ____________ |
| Implementation | Priority: ______ Pass: __________________ | Migration rehearsal, rollback or export Your detail: ____________ | Level: __ / 3 Evidence: ______________ | Status: _________________ Gap / owner: ____________ |
Traceability
Test the investigation, not just lot capture.
The Food Standards Agency's quick reference guide says food businesses need supplier and business-customer traceability, commonly described as one step back and one step forward, and systems that can make traceability information available to enforcement authorities on demand.
Ask the vendor to start with a source lot, follow it through a split or production event and identify dispatches. Then reverse the exercise from a customer delivery. Include quarantine, relabelling, partial quantities and missing information so you see how exceptions are surfaced.
Read the FSA traceability guideA useful traceability demo should show
- Supplier, goods-receipt and customer-dispatch connections
- Parent-child lineage after processing, splitting or repacking
- Dates, origin, allergens and other product-specific details
- Hold, quarantine, recall-lock and release responsibilities
- A clear export or evidence view for an investigation
- What happens when a required record is incomplete
Separate records from food-safety responsibility
The FSA's MyHACCP tool describes a food-safety management system based on HACCP principles. Ask how the ERP records checks, limits, failures, corrective action and sign-off, but keep ownership of hazard analysis and procedures with competent people in your business.
Check allergen change control
FSA guidance emphasises accurate, up-to-date allergen information and the effect of recipe or ingredient changes. Test who can change product and recipe data, what is reviewed and how labels or customer information are updated.
Make variable weight commercial
A catch-weight demo should reach the final value, not stop at the scale. Test target versus actual, tolerance handling, user or device provenance, per-kilogram pricing, labels, dispatch and any credit or invoice consequence.
Security and supplier diligence
Match evidence to the impact of failure.
NCSC guidance recommends building confidence that a cloud provider is secure enough for the intended data and use. For an operational ERP, consider confidentiality, integrity and availability: a leak, incorrect stock record or outage can each matter differently.
Customer separation
Ask the provider to explain the boundaries that prevent one customer from accessing or affecting another customer's service and data, including inherited cloud controls.
Audit information
Establish which security and operational actions are logged, how records are made available, their format and retention, and whether the evidence meets your incident needs.
Claims and assurance
Ask for evidence behind public assertions, understand shared responsibilities and decide what level of independent validation is proportionate to your risk.
Recovery evidence
Prove you can restore the records you rely on.
The NCSC's cloud backup principles call for a known-good restore and a way to test the current backup state without disrupting live infrastructure. For an ERP, choose a small, representative chain of records and ask the supplier which parts can be recovered together. This is a diligence exercise, not a claim that any particular system has passed it.
1. Define the loss
Agree a scenario such as accidental deletion or corruption. Name the last acceptable recoverable point and how long the operation can work without the system; use your own business limits.
2. Inventory the evidence
List a supplier lot, its stock movements, a customer order, dispatch or delivery evidence, invoice-ready values, attached files and the configuration needed to interpret them. Ask what is included, excluded or backed up separately.
3. Demonstrate in isolation
Ask for a restore into a separate environment. Check that the chosen records can be opened, their quantities and links still agree, and the corresponding files can be read. A backup job marked successful does not prove this result.
4. Record the decision
Compare the result with your agreed recoverable point and time. Write down the backup version, time to usable readback, missing records or files, who can invoke recovery, and the fallback while work is interrupted. Assign an owner and retest date to each gap.
If a supplier cannot show an isolated restore, record exactly what evidence was available and what remains unverified. Decide whether that gap is acceptable before depending on the system for live work.
Read the NCSC backup principlesImplementation and exit
Buy the delivery plan as carefully as the software.
Ask for the first operational phase, named owners, data required, migration rehearsals, user acceptance tests, training and go-live decision criteria. Keep “available today”, “configuration”, “integration”, “custom build” and “roadmap” as separate columns.
Also agree how data can be exported, how long it is retained and what assistance is available if the relationship ends. An exit plan is easier to agree before the system becomes operationally critical.
Use the parallel-run worksheet in our modernisation playbookTwelve questions to take into procurement
- 1.Which operational record is the source of truth at each stage?
- 2.Can a lot be traced both forwards and backwards after splitting or repacking?
- 3.How are quarantine, recall locks and failed checks prevented from becoming normal stock?
- 4.Can actual weight drive the final line value, and who can override it?
- 5.How are shortages, substitutions, credits and delivery failures represented?
- 6.What POD evidence can be required, and what happens if it cannot be uploaded?
- 7.Which accounting paths exist today, and which would be implementation work?
- 8.How are customer data, roles and privileged support access controlled?
- 9.What audit information is available, for how long and in what format?
- 10.How will master data be cleaned, mapped, rehearsed and accepted?
- 11.What happens if a go-live phase must be paused or rolled back?
- 12.How can we retrieve our data if we leave the service?
Primary sources used
These sources inform the evaluation questions; they do not turn this guide into legal advice. Check the latest guidance for your location and operation.
- Food Standards Agency: food traceability, withdrawals and recalls quick reference guide
- Food Standards Agency: MyHACCP
- Food Standards Agency: allergen guidance for food businesses
- NCSC: choosing a cloud provider
- NCSC cloud security principle 3: separation between customers
- NCSC cloud security principle 13: audit information and alerting
- NCSC: principles for ransomware-resistant cloud backups