WCPS1.5_WCPS Interval expression to actually support mathematical expressions

[Piero Campalani]

Currently, ANTLR breaks when trying to put expressions into an interval expression.

Toy example:

for c in (eobstest) return encode (
   coverage averaged_pixel_history
   over     $t t(0:(1+1))
   values   avg(c[Lat(0), Long(51)])
, "csv" )

Use case:

for c in (eobstest) return encode (
   coverage averaged_pixel_history
   over     $t t(0:(count(c[Lat(0), Long(51)]<100000)/2))
   values   avg(c[Lat(0), Long(51), t:"CRS:1"($t:$t+1)])
, "csv" )

Relevant grammar:

axisIterator:
    variableName axisName ( intervalExpr )

intervalExpr:
    integerExpr : integerExpr
  | imageCrsDomain ( coverageName , axisName )

Relevant Petascope grammar (IntegerExpr = IndexExpr):

indexExpr returns[IndexExpr value]                                                                                                                           
    : e1=indexTerm { $value = $e1.value; } 
                (op=(PLUS|MINUS) e2=indexTerm { $value = new IndexExpr($op.text, $value, $e2.value); })* 
    ;   

indexTerm returns[IndexExpr value]
    : e1=indexFactor { $value = $e1.value; }
        ((op=(MULT|DIVIDE) e2=indexFactor { $value = new IndexExpr($op.text, $value, $e2.value); } ))* 
    ;   

indexFactor returns[IndexExpr value]
    : e=INTEGERCONSTANT  { $value = new IndexExpr($e.text); }
    | op=ROUND LPAREN e1=numericScalarExpr RPAREN  { $value = new IndexExpr($op.text, $e1.value); }
    | (LPAREN e2=indexExpr RPAREN { $value = $e2.value; } ) 
    ;   

[Piero Campalani]

Ticket #360 can be related or overlapping.

[Vlad Merticariu]

The WCPS subsets are transformed into rasdaman indexes in petascope. For this reason, petascope needs to know the actual subset values for computing the equivalent rasql query.
E.g. if Lat(100-50) is given, petascope needs to know that this actually means Lat(50) in order to compute the equivalent rasdaman index. A more complicated one would be Lat(count(mean_summer_airtemp)). count(mean_summer_airtemp) needs to be evaluated by rasdaman, the result returned to petascope, where it would be transformed into an index.

This means that, in order to support this feature, we need to change the model of petascope and have a 2 step evaluation process: first evaluate all the indexes, then use the results to obtain a rasql query equivalent to the WCPS request.

[Dimitar Misev]

Just an idea, maybe the Lat() function is not so complicated and can be translated into rasql?

[Vlad Merticariu]

Yes, that is a valid approach as well, in my opinion.

However, in order for this to work, we need to port to rasdaman the whole CRS handling. This would allow petascope(WCPS) to act as a parser and translate everything to a rasql query in 1 step.

I am not sure, however, how easy is to do CRS handling in rasdaman (E.g. mr[0:100,0:100] to be transformed into mr[100:200,100:200] if mr is in some CRS). This would lead to geo-referenced arrays in rasdaman.

[Vlad Merticariu]

Sorry, not necessarily geo-referenced, but tied to a CRS.

[Dimitar Misev]

That's not exactly what I had in mind. Usually you just need the origins and resolution, and via some formula you can translate the CRS coordinate to an index coordinate. I assume this is done in CrsUtil.convertToInternalGridIndices, which looks quite elaborate..

My suggestion was to put the whole formula in the final rasql query instead of executing it in petascope, this doesn't require rasdaman to know of any geo-referencing. But I'm not sure how complicated the formula would get and it would be unfeasible if we use some libraries for this.

[Vlad Merticariu]

I'm not sure that this would work in practice. Basically what we would need to do is to write the body of the CrsUtil.convertToInternalGridIndices in rasql and somehow provide all the context information as well.

[Dimitar Misev]

Usually it's something like

Lat(X) = (X / resolution) + origin

which is not too complicated, I've used this in bash scripts to automate import for example. So it should be as simple as resolving the resolution and origin from petascopedb/CRS definition. But I may be missing something not sure, it's just another possibility

[Piero Campalani]

Dimitar,
that is the formula on regular axes, whereas you need to fetch those coefficients from petascopedb for irregular ones.

Keep in mind also that in case there will be the need to "free" rasdaman from *aligned* geo-referenced data, a huge refactoring of subsets handling is needed in Petascope, which is also why the support for warped/rotated coverages was postponed for Earthserver: now subsets are solved one at a time, while a lazier subsets converter (geo2rasql, I mean) is necessary to convert more complex unaligned subsettings to the regularly gridded world of marrays. Policies on posterior masks or minimal-bbox should also be considered, but this is becoming unrelated to this ticket. hth

[Vlad Merticariu]

Changed milestone to 10. This will be fixed in WCPS 2, when it will be made default.

[Bang Pham Huu]

This is also my concern because it is impossible if try to nested the value from reduced expression into interval (i.e, Vlad's comment above), especially in geo-referenced axis. I think it is the same as in #1259 and we could not support this kind of functionality as in here (a discussion will confirm later).

[Dimitar Misev]

Any updates on this ticket?