encode to csv and json outputs rounded values for float and double, but not always

[bbell]

creating a structure of type {float, double, octet, double, short}, and ingesting a file with 00000001 for every byte (such a file is attached) yields rounded-off values for floats and doubles when using encode with the "csv" or "json" parameter, but inconsistently (the first value of the first cell is not rounded).

suppose you have inserted the data into a collection named "test_struct"

rasql -q 'select encode(c, "json") from test_struct as c' --out string

has an output of the form:

$ rasql -q 'select encode(c,"json") from test_struct as c' --out string
rasql: rasdaman query tool v1.0, rasdaman 9.4.0.
opening database RASBASE at localhost:7001...ok
Executing retrieval query...ok
Query result collection has 1 element(s):
  Result object 1: [[["2.36943e-38 0.000000 1 0.000000 257"],["0.000000 0.000000 1 0.000000 257"],["0.000000 0.000000 1 0.000000 257"],["0.000000 0.000000 1 0.000000 257"],

(repeats with 2.36943e-38 never appearing again...)

It is worth remarking that the double value for the following 8 bytes:

00000001 00000001 00000001 00000001 00000001 00000001 00000001 00000001

is 7.7486e-304

Most likely, this problem has to do with precision handling in csv/json output.

[bbell]

binary file with every byte 00000001

[bbell]

The culprit here is the use of "fixed" on line csv.cc:208

    case r_Type::DOUBLE:
        f << fixed << static_cast<double>(ptr->get_double(val));

Dropping that stream operation yields the expected outputs; however, is this a feature rather than a flaw?

fixed is coming from ios_base.h:

  // [27.4.5.4] floatfield manipulators
  /// Calls base.setf(ios_base::fixed, ios_base::floatfield).
  inline ios_base&
  fixed(ios_base& __base)
  {
    __base.setf(ios_base::fixed, ios_base::floatfield);
    return __base;
  }

[Dimitar Misev]

Perhaps check with git whatchanged first if there's some particular patch that put fixed there.

In any case, I'm also not sure whether it makes more sense with fixed or without.

[Dimitar Misev]

But removing it will certainly be a breaking change.

[bbell]

Agreed that it would not be a minor change. One other thing I noticed is that an r_Double is a typedef for double, and we are also throwing a static_cast<double> there where it (should?) not be needed.

What I can do for now is try to find out why the first element of the first struct is treated a bit differently (I suspect it's viewed as a float until the struct's first double is encountered and supersedes, because "fixed" is only being used on doubles, not on floats). I'll look into it a bit more and update again here.

In any case, should the behaviours of float and double both be the same for json/csv output?

[bbell]

After discussion at yesterday's meeting, we will implement the change to be consistent for both float and double, and in particular, the output will reflect the previous behaviour of floats (i.e. no use of 'fixed' via ios_base.h)

[Dimitar Misev]

Should it be closed?

[bbell]

Replying to dmisev:

Should it be closed?

I think so.