Calculation Issues

How to Fix Excel and VBA Returning Different COS Values

Aamir Naveed Akram Sep 30, 2026 868 views

Question details

The user wants to understand and resolve the discrepancy in trigonometric calculation results between Microsoft Excel and VBA.

Why Excel and VBA Return Different COS Values

Product: Microsoft Excel, VBA
Device & OS: not provided
Scenario: Calculating trigonometric functions such as COS of 90 degrees using both Excel worksheet formulas and VBA scripts.
Observed behavior: Excel and VBA return slightly different calculation results (e.g., extremely small numbers like 6.12E-17 instead of 0) due to floating-point arithmetic precision limits and internal calculation differences.

Before you start

Before adjusting your calculations, confirm the exact decimal precision required for your project, as floating-point variations are normal in all modern spreadsheet software.

Solution 1Recommended

Use the Round Function to Standardize Results

Rounding the output resolves floating-point discrepancies, standardizing the values especially when the expected trigonometric result should be exactly zero.

Floating-point arithmetic limitations mean that computers sometimes cannot accurately represent absolute zero in trigonometric functions like COS of 90 degrees. Both Excel and VBA have slightly different internal implementations for math calculations.

To obtain consistent results across both platforms, apply a uniform rounding function. This ensures that functionally equivalent values are displayed and treated identically.

Locate the VBA calculation

Open your VBA editor by pressing Alt + F11, and locate the specific script where the COS function is being used.

Apply rounding in VBA

Wrap your COS calculation with the VBA Round function. For example, change your code to: Result = Round(Cos(angle), 10) to limit the precision to 10 decimal places.

Apply rounding in Excel

Go to your Excel worksheet and update your cell formula to match the VBA precision. Use a formula like: =ROUND(COS(A1), 10).

Compare the standardized values

Run your VBA macro and check your Excel worksheet to verify that both calculations now output identical, meaningful numbers (e.g., exact 0).

Use the Round Function to Standardize Results

Scientific Notation Awareness: Values such as 6.12E-17 represent extremely small numbers close to zero. This is standard behavior for floating-point calculations governed by IEEE 754 standards, not a software bug.

Perform Accurate Calculations with WPS Spreadsheet

WPS Spreadsheet provides a robust calculation engine fully compatible with Microsoft Excel formulas. You can easily manage floating-point precision and apply ROUND functions to ensure your data remains perfectly accurate.

1. Install WPS Office: Download and install WPS Office, then launch WPS Spreadsheet.
2. Input your data: Enter your numerical data or angle measurements into your desired cells.
3. Apply calculation formulas: Use the formula =ROUND(COS(RADIANS(A1)), 10) to calculate a precise, rounded trigonometric result.
4. Run compatible macros: Open the built-in Macro editor in WPS Spreadsheet to write or execute your existing VBA code with identical precision control.

Fully compatible with Microsoft Excel .xlsx formats and complex VBA macrosPrecise mathematical and trigonometric function supportFree and lightweight spreadsheet alternative with a familiar user interfaceSeamless migration of your existing formulas and datasets

microsoft office alternative - wps office

Frequently Asked Questions

Can I use WorksheetFunction.Cos in VBA to fix the difference?

No, VBA does not support WorksheetFunction.Cos because VBA already has its own built-in Cos function. You must use the native VBA Cos function and manually apply rounding to match the Excel worksheet's output.

Why do computers struggle with floating-point math?

Computers use binary hardware to represent decimal numbers, which can lead to minor precision errors, such as returning 6.12E-17 instead of an exact 0. This is governed by the universal IEEE 754 standard used by most modern software, including Excel and VBA.

Is the COS calculation discrepancy considered an Excel bug?

No, this minor calculation difference is not a software bug. It is a known limitation of floating-point arithmetic. Both Excel and VBA calculate results correctly within their respective precision limits.

Excel & Data Problems