diff options
| author | Peter Zhu <peter@peterzhu.ca> | 2024-07-08 19:32:45 -0400 |
|---|---|---|
| committer | Peter Zhu <peter@peterzhu.ca> | 2024-07-09 08:55:58 -0400 |
| commit | 5de6d0b35f17520ada1c2e9433022827b14d780e (patch) | |
| tree | da9e462fddf18a76b9668b53e850a0d4505bc788 /numeric.c | |
| parent | ad6b2e89851dd31f10f91aca408681ba39c23ede (diff) | |
[DOC] Fix granularity calculation
The granularity is calculated as `10 ** ndigits.abs` rather than `ndigits.abs * 10`. For example, if `ndigits` is `-2`, it should be `10 ** 2 == 100` rather than `2 * 10 == 20`.
Diffstat (limited to 'numeric.c')
| -rw-r--r-- | numeric.c | 4 |
1 files changed, 2 insertions, 2 deletions
@@ -5722,7 +5722,7 @@ int_round(int argc, VALUE* argv, VALUE num) * - When `self` is non-zero and `ndigits` is negative, * returns a value based on a computed granularity: * - * - The granularity is <tt>ndigits.abs * 10</tt>. + * - The granularity is <tt>10 ** ndigits.abs</tt>. * - The returned value is the largest multiple of the granularity * that is less than or equal to `self`. * @@ -5791,7 +5791,7 @@ int_floor(int argc, VALUE* argv, VALUE num) * - When `self` is non-zero and `ndigits` is negative, * returns a value based on a computed granularity: * - * - The granularity is <tt>ndigits.abs * 10</tt>. + * - The granularity is <tt>10 ** ndigits.abs</tt>. * - The returned value is the smallest multiple of the granularity * that is greater than or equal to `self`. * |
