- 1.
- Workstation version of the RAP code using the optimized
floating-point vector/matrix library from ICSI (fltvec). [September
1996]
- 2.
- SPERT version using the fixed-point vector/matrix library (fxlib).
This version performs all arithmetic in 16-bit fixed-point.
Lack of precision caused unstable training and the acoustic models to
result in approximately 100% increase in word error rate. [January 1997]
- 3.
- Fixed-point arithmetic changed to 32-bit where supported by the
hardware and the fixed-point library. This failed to solve the
problems seen in the 16-bit version. [February 1997]
- 4.
- Those areas where both dynamic range and precision were considered
important were changed to floating-point. This solved the stability
problem, but training was very slow due to the need to do
floating-point emulation (SPERT does not have hardware support for
floating-point). [April 1997]
- 5.
- A complete re-write of the RNN training software to use data
structures more suitable for fixed-point arithmetic (separate data
items for all values whose ranges and/or required precision are
different). This was initially written using in floating-point to run on
workstations. [June 1997]
- 6.
- Converted new version into 16-bit fixed-point. Still had problems with
stability during training. A 32-bit version was also written, but this
did not solve the stability problem. The underlying problem with all
the fixed-point versions is the need to accumulate a large number of
small gradients. This requires the ability to accurately represent
small values and also a large dynamic range. ICSI provided direct
assistance with setting of exponents and in the use of a mix of 16 and
32-bit arithmetic. [August 1997]