GuidesPrecision in Engineering Conversions
Precision in Engineering Conversions
How engineers should convert units without destroying tolerances—significant figures, exact factors, stack-ups, and documentation habits that keep parts fitting.
Tolerances do not survive careless conversion
A shaft specified as 10.00 ± 0.02 mm is not the same as casually writing 0.394 in after a phone conversion rounded to three decimals. Soft conversion can eat the tolerance budget before the machine starts. The scrap bin fills with parts that were converted casually rather than machined poorly.
Hard conversion uses exact factors and then rounds according to a stated rule that preserves the intent of the tolerance. Soft conversion picks a convenient nearby nominal in the other system. Both are valid strategies—but they are different strategies. Write which one you used in the revision block.
Engineering drawings should declare the conversion policy. Global teams that mix methods produce parts that almost fit, which is worse than parts that clearly do not. Almost-fitting parts waste inspection time and erode trust between suppliers.
Exact factors and significant figures
Use exact definitions: 1 in = 25.4 mm. Convert with full precision, then round once for the drawing. Intermediate rounding in a multi-step spreadsheet is a common source of mysterious 0.01 mm disagreements between vendors. Lock conversion cells as formulas, not pasted values, when possible.
Significant figures should reflect measurement capability and functional need. Listing eight decimals after converting a tape-measure dimension pretends to precision you do not have. Inspectors should be told which digits are functional and which are artefacts of conversion display.
For statistical process control, keep one internal unit system in the database and convert only at presentation boundaries. Converting stored history repeatedly degrades data and complicates capability studies across plants.
Bilateral tolerances and unit changes
When converting a nominal with ± tolerance, convert the limits (upper and lower) and then restate nominal and tolerance, or convert nominal and tolerance with consistent rounding rules that never shrink the allowed zone unintentionally.
Unequal tolerances and fits (H7/g6 style thinking, even when expressed in inches historically) need special care. Convert the limiting sizes that define the fit, not just a single mid value.
GD&T frames and datum schemes are unit-agnostic in logic but unit-specific in numbers. Update the entire feature control frame set coherently.
Area, volume, and derived quantities
Stress, pressure, density, and flow combine units. Convert each base dimension with factors raised to the correct powers. A pressure conversion error often starts as a length or area factor used linearly.
Finite-element and CFD models are merciless about unit consistency. Pick SI for the model, convert inputs in, convert outputs out. Mixed-unit decks are legendary failure modes.
Bill-of-materials lengths might be metric while packaging volumes are litres and shipping weights are kilograms—still keep each quantity’s conversions isolated and labelled.
Software, PLM, and dual dimensioning
CAD dual dimensioning should derive the secondary unit from the primary, never the reverse on alternate days. Primary units own the design authority. Secondary values are communication aids, not a second source of truth.
PLM exports to suppliers must state units in the file, not only in an email. Ambiguous CSV columns are scrap tickets waiting to happen.
Automated converters in ERP systems need decimal arithmetic for money-adjacent quantities and for regulated dimensions. Binary float leftovers show up as mismatched hashes between plants.
A field checklist
Identify authoritative units. Choose hard or soft conversion deliberately. Use exact factors. Convert limits for fits. Round once. Record the policy. Verify with a go/no-go mental estimate.
On the shop floor, give machinists drawings in the unit their tools display. Forcing constant mental conversion increases scrap more than any factor-table debate. If dual dimensions are required, derive the secondary unit from one authoritative primary value every time the drawing updates.
When a part fails inspection after a unit change, audit the conversion before auditing the operator. Many escapes are metrological, not motivational. Keep a short conversion verification coupon on first-article inspections whenever drawings change unit systems.
Related converters
Frequently asked questions
What is the difference between hard and soft conversion?
Hard conversion uses exact factors and preserves the original intent as closely as rounding allows. Soft conversion chooses a convenient near value in the other unit system, possibly adjusting tolerances, for manufacturability in local tooling.
Should I round millimetre equivalents of inches to whole millimetres?
Only for soft conversion where the design can tolerate it. For precision fits, keep the necessary decimals after multiplying by 25.4 and round according to a documented rule that protects the tolerance zone.
Why do two vendors disagree in the fourth decimal place?
Often because of intermediate rounding, different foot definitions historically, or floating-point display modes. Agree on exact factors, a single conversion path, and a shared rounding policy.