Opened 9 years ago
Closed 9 years ago
#1231 closed defect (invalid)
Rasql_Wrong negative value in calculation
Reported by: | Bang Pham Huu | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | 10.0 |
Component: | undecided | Version: | development |
Keywords: | WCPS, wrong value, Rasql, negative number | Cc: | Dimitar Misev |
Complexity: | Medium |
Change History (3)
comment:1 by , 9 years ago
comment:2 by , 9 years ago
Sorry, actually the query does not need encode in "unsigned char", example below
for c in (rgb) return encode( (c.red - 248)[i(0:0), j(0:0)] , "csv")
will return {-129}
but
for c in (rgb) return encode( (c.red - c.blue)[i(0:0), j(0:0)] , "csv")
will return {127}
and I expected it should be negative number.
comment:3 by , 9 years ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
The original types are unsigned char of c.red and c.blue, the minus operation doesn't automatically expand the types. You should cast both operands to short for example for this exapmle to work.
for c in (rgb) return encode( ((short)c.red - (short)c.blue)[i(0:0), j(0:0)] , "csv")
Note:
See TracTickets
for help on using tickets.
But you do a cast to (unsigned char), what should be the proper result?