Number formats

There are multiple number formats among the protocols and formats around CASIO calculators. The following sections describe some of them.

fx-9860G-style BCD

Numbers with the fx-9860G and compatible are represented using 12 bytes, i.e. 24 nibbles, with only the first 9 bytes, i.e. 18 nibbles, being significant.

Assuming we use 0-based indexing:

  • Nibbles 0 to 2 represent the exponent and sign, with the following ranges:

    Raw range

    Sign

    Real range

    000 - 499

    +

    -99 - 399

    500 - 999

    -

    -99 - 399

  • Nibbles 3 to 17 represent the mantissa, starting from 10^0.

Examples are the following:

Raw number (1 digit / nib.)

Sign

Exponent

Mantissa

Result

600 123000000000000 xxxxxx

-

1 (600 - 100 - 99)

.123

-1.23

100 230400000000000 xxxxxx

+

1 (100 - 99)

.2304

2.304

098 456000000000000 xxxxxx

-

-1 (98 - 99)

.456

.0456

597 786000000000000 xxxxxx

-

-2 (597 - 500 - 99)

.786

-.00786

fx-9860G-style fraction

Fractions with the fx-9860G and compatible are represented using 12 bytes, i.e. 24 nibbles, with only the first 9 bytes, i.e. 18 nibbles, being significant.

Assuming we use 0-based indexing:

  • Nibble 0 represents the sign (1 for positive, 6 for negative);

  • Nibble 1 is always A;

  • Nibble 2 is the number of nibbles to read after, minus 1.

Examples are the following:

Raw number (1 digit)

Sign

a

b

c

Result

6A 6 2 A 33 A 50 00000000 xxxxxx

-

2

33

50

-2 * (33 / 50)

1A 7 2 A 33 A 500 0000000 xxxxxx

+

2

33

500

2 * (33 / 500)