Skip to content

Commit

Permalink
optimizing d2string() and addReplyDouble() with grisu2: double to str…
Browse files Browse the repository at this point in the history
…ing conversion based on Florian Loitsch's Grisu-algorithm (redis#10587)

All commands / use cases that heavily rely on double to a string representation conversion,
(e.g. meaning take a double-precision floating-point number like 1.5 and return a string like "1.5" ),
could benefit from a performance boost by swapping snprintf(buf,len,"%.17g",value) by the
equivalent [fpconv_dtoa](https://github.com/night-shift/fpconv) or any other algorithm that ensures
100% coverage of conversion.

This is a well-studied topic and Projects like MongoDB. RedPanda, PyTorch leverage libraries
( fmtlib ) that use the optimized double to string conversion underneath.


The positive impact can be substantial. This PR uses the grisu2 approach ( grisu explained on
https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf section 5 ). 

test suite changes:
Despite being compatible, in some cases it produces a different result from printf, and some tests
had to be adjusted.
one case is that `%.17g` (which means %e or %f which ever is shorter), chose to use `5000000000`
instead of 5e+9, which sounds like a bug?
In other cases, we changed TCL to compare numbers instead of strings to ignore minor rounding
issues (`expr 0.8 == 0.79999999999999999`)
  • Loading branch information
filipecosta90 authored Oct 15, 2022
1 parent 9ab873d commit 29380ff
Show file tree
Hide file tree
Showing 18 changed files with 662 additions and 23 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ deps/lua/src/lua
deps/lua/src/luac
deps/lua/src/liblua.a
deps/hdr_histogram/libhdrhistogram.a
deps/fpconv/libfpconv.a
tests/tls/*
.make-*
.prerequisites
Expand Down
7 changes: 7 additions & 0 deletions deps/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ distclean:
-(cd lua && $(MAKE) clean) > /dev/null || true
-(cd jemalloc && [ -f Makefile ] && $(MAKE) distclean) > /dev/null || true
-(cd hdr_histogram && $(MAKE) clean) > /dev/null || true
-(cd fpconv && $(MAKE) clean) > /dev/null || true
-(rm -f .make-*)

.PHONY: distclean
Expand All @@ -66,6 +67,12 @@ hdr_histogram: .make-prerequisites

.PHONY: hdr_histogram

fpconv: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd fpconv && $(MAKE)

.PHONY: fpconv

ifeq ($(uname_S),SunOS)
# Make isinf() available
LUA_CFLAGS= -D__C99FEATURES__=1
Expand Down
23 changes: 23 additions & 0 deletions deps/fpconv/LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
Boost Software License - Version 1.0 - August 17th, 2003

Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:

The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
27 changes: 27 additions & 0 deletions deps/fpconv/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
STD=
WARN= -Wall
OPT= -Os

R_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS)
R_LDFLAGS= $(LDFLAGS)
DEBUG= -g

R_CC=$(CC) $(R_CFLAGS)
R_LD=$(CC) $(R_LDFLAGS)

AR= ar
ARFLAGS= rcs

libfpconv.a: fpconv_dtoa.o
$(AR) $(ARFLAGS) $@ $+

fpconv_dtoa.o: fpconv_dtoa.h fpconv_dtoa.c

.c.o:
$(R_CC) -c $<

clean:
rm -f *.o
rm -f *.a


9 changes: 9 additions & 0 deletions deps/fpconv/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
libfpconv

----------------------------------------------

Fast and accurate double to string conversion based on Florian Loitsch's Grisu-algorithm[1].

This port contains a subset of the 'C' version of Fast and accurate double to string conversion based on Florian Loitsch's Grisu-algorithm available at [github.com/night-shift/fpconv](https://github.com/night-shift/fpconv)).

[1] https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf
Loading

0 comments on commit 29380ff

Please sign in to comment.