How to Read an Instant Measurement Accuracy Spec: Accuracy, Repeatability and the ± Convention

Direct answer: three different numbers on an instant measurement specification sheet get confused with each other. Measurement accuracy is the stated deviation from true value under stated conditions — it is the number you compare with your tolerance. Repeatability describes the spread when the same feature is measured repeatedly — it is the number you compare with your gauge-capability requirement. Minimum display unit is only a resolution figure and says nothing about accuracy. Every ± value is valid only inside the range condition printed next to it, and only under the environmental conditions stated in the same document.

The Three Numbers, Separated

TermWhat it answersHow to use it
Measurement accuracy (±x µm within A × B mm)How close to the true value, under stated conditionsCompare with your tolerance band
Repeatability (±x µm)How stable the result is when you measure againCompare with your measurement-capability (gauge) requirement
Minimum display unit (0.1 µm)Display resolutionNever use as an accuracy claim

Taking the POMEAS IMAGE 3 series as a worked example (specification 20250805, page 30): repeatability is stated as 0.5 µm in high-precision mode and 1 µm in wide-field mode for IMAGE 3, PLUS, PRO and PRO-H, and 0.25 / 0.5 µm for MAX and MAX 2. Measurement accuracy in the no-stitching (single-field) mode is stated as ±2 µm within 20 × 14 mm for the four 20 MP models.

Why the Range Condition Matters

"±2 µm" on its own is not a specification. The IMAGE 3 sheet states ±2 µm within 20 × 14 mm — that is the single-field, no-stitching condition. If your part is larger than that field, the system stitches multiple fields and the accuracy statement changes to a formula: ±(2+0.02L) µm within 113 × 105 mm for IMAGE 3 PLUS, where L is the stage travel distance. The farther the stage moves, the larger the term. A specification quoted without its range condition is incomplete, and comparing two vendors on the bare ± number is meaningless.

Arithmetic: Turning Two ± Numbers Into a Ratio

Because both the tolerance and the machine capability are written as ± values, the comparison must be made on the full interval, which is 2 × |± value| for each side. Example with IMAGE 3 MAX 2 in high-precision mode (accuracy ±1 µm within 5 × 5 mm, repeatability ±0.25 µm) against a drawing tolerance of ±10 µm:

  • Tolerance interval: 2 × 10 µm = 20 µm
  • Machine accuracy interval: 2 × 1 µm = 2 µm → tolerance-to-accuracy ratio 20 ÷ 2 = 10 : 1
  • Machine repeatability interval: 2 × 0.25 µm = 0.5 µm → tolerance-to-repeatability ratio 20 ÷ 0.5 = 40 : 1

Both ratios are useful, but they answer different questions: the first asks whether the machine can resolve your tolerance, the second whether the measurement itself is a stable basis for a pass/fail decision. Which ratio your quality system requires is your own acceptance criterion — it is not a number a vendor should decide for you, and it is not a figure we will state on your behalf.

A common error is to halve or double one side only: if you divide 20 µm by 0.25 µm you get 80, which is wrong, because a ±10 µm tolerance and a ±0.25 µm capability are both two-sided intervals.

Conditions the Numbers Are Attached To

The IMAGE 3 specification states common test conditions that the accuracy figures assume:

  • Temperature 20 ± 1 °C
  • Humidity 30–80 %, non-condensing
  • Vibration below 0.002 g at 15 Hz
  • Stage load 2 kg or less
  • Minimum display unit 0.1 µm
  • No-stitching mode verified with an optical calibration chart; stitching mode verified with a line scale

On a shop floor at 28 °C next to a stamping press, none of these hold, and the stated numbers do not apply as printed. This is not a defect of the instrument — it is what "under stated conditions" means. Validate on your part, in your environment, before writing the numbers into a control plan.

What to Do With a Blank Cell

The IMAGE 3 series sheet has cells the specification simply does not fill in. Two examples: the high-precision no-stitching accuracy of IMAGE 3 MAX (value ±1 µm stated, range condition not stated — pending engineering confirmation), and the wide-field no-stitching accuracy of IMAGE 3 MAX 2 (cell empty). A blank cell means unknown. It must not be filled with a value borrowed from a neighbouring model, and it must not be presented to a customer as "supported". The correct output is: not stated, pending confirmation.

Five Ways the Sheet Gets Misread

  1. Using repeatability as accuracy. 0.25 µm repeatability does not mean ±0.25 µm accuracy.
  2. Dropping the range condition. ±2 µm within 20 × 14 mm is not ±2 µm across a 200 mm stage travel.
  3. Ignoring L. In the stitching formula ±(2+0.02L) µm, a 100 mm stage travel adds 2 µm.
  4. Ignoring the environment. The stated conditions are part of the number.
  5. Quoting a calculated ratio as a measurement result. Ratio arithmetic is a screening tool; acceptable capability must be demonstrated on the actual part.

FAQ

What is the difference between measurement accuracy and repeatability?

Accuracy states closeness to the true value under stated conditions; repeatability states the spread between repeated measurements of the same feature. Accuracy is compared with tolerance, repeatability with gauge capability.

Why is the accuracy written as a formula with L?

In stitching mode the stage moves between fields, so total error grows with travel distance. L is the stage travel; for IMAGE 3 PLUS the statement is ±(2+0.02L) µm within 113 × 105 mm.

Is 0.1 µm minimum display unit the same as 0.1 µm accuracy?

No. Display resolution is not an accuracy statement and must never be quoted as one.

Can I compare two machines on the ± number alone?

Only if the range condition, the mode (stitching or not), and the test conditions are identical. Otherwise you are comparing numbers that do not measure the same thing.

What if the part is bigger than the stated accuracy range?

Use the stitching-mode formula and add the L term, or move to a model whose single field covers the part.

Do the numbers hold on the shop floor?

They hold under the stated conditions (20 ± 1 °C, humidity 30–80 %, vibration below 0.002 g at 15 Hz, stage load ≤ 2 kg). Outside those conditions, validate on your part.

What does an empty cell in the specification mean?

Unknown. It is recorded as "not stated, pending confirmation" and must not be inferred from another model.

How do I state a capability ratio for my own part?

Convert both sides to full intervals (2 × |± value|) and divide. Whether the resulting ratio is acceptable is your quality system's decision, supported by a measurement on your part.

Related Tools and Next Step

Have the Numbers Checked Against Your Drawing

Send the drawing with the tolerance list and we will state, per dimension, which IMAGE 3 mode and model covers it, what the applicable accuracy statement is, and which cells remain unknown pending confirmation — in writing, before evaluation.

Go Back Top
VK Message
WhatsApp

Scan QR Code

WhatsApp QR Code
Wechat

Scan QR Code

Wechat
Phone Number
+8618598102007
Copied!
Online Message

Online Message

Click to refresh