Rasql_Auto round floating point value can make wrong result

[Bang Pham Huu]

WCPS query

for c in (mr) return encode(  pow( c[i(100:110),j(100:110)], 0.5)  + 200000 , "csv" )

will returns {200013, 200014, 200015,…}

but it should returns the correct floating points.

for c in (mr) return encode(  pow( c[i(100:110),j(100:110)], 0.5)  , "csv" )

which returns

{14.3178,14.2127,14.0712,13.9284,13.8924,13.8564,13.7477,13.7477,13.8203,13.8924,13.8203}

then when + 200000 it should be {200014.3178, 2000014.2127, 2000013.8924,…}

The problem is with this behavior, when using statistic the data is "fixed" and can result untrusted result, e.g counting point values > 2000014.200 (suppose has 100 points) but because 14.200 is converted to 14 (and + 200000 to 200014) then only returns 10 points.

[Dimitar Misev]

Why rasql, have you tested if the issue is in rasql actually? What is the rasql query?

[Bang Pham Huu]

Dimitar, Rasql query

select csv(((pow(c[100:110,100:110], 0.5))+(200000))) from mr AS c where oid(c)=45569

And tested

rasql -q "select csv(((pow(c[100:110,100:110], 0.5))+(200000))) from mr AS c where oid(c)=45569" --out string
rasql: rasdaman query tool v1.0, rasdaman v9.2.0-beta1-gf79e291 -- generated on 21.01.2016 14:39:26.
opening database RASBASE at localhost:7001...ok
Executing retrieval query...ok
Query result collection has 1 element(s):
  Result object 1: 

{200013,200013,200013,200013,200013,200013,200013,200013,200013,200014,200014},...
rasql done.

[Dimitar Misev]

It seems like a bug in the csv function in particular, exporting to tiff or netcdf for example gives the right values. min_cells/max_cells also gives wrong values.

[Bang Pham Huu]

thanks for updating

[Dimitar Misev]

rasql -q 'select netcdf(((pow(c[100:110,100:110], 0.5))+(200000.0))) from test_grey2 AS c' --out file
ncdump rasql_1.nc

This will show correct values. Probably it's some precision issue with C++ streams which csv uses.