inv_tiff segmentation

[Piero Campalani]

In some machines it is not possible anymore to ingest TIFF ot GeoTIFF files, due to a compatibility problem for the inv_tiff() function.

In nohup I get:

rasserver: tif_open.c:449: TIFFClientOpen:
Assertion `(toff_t)tif->tif_size==n' failed.

My system:

$ lsb_release -a
LSB Version:	core-2.0-ia32:core-2.0-noarch:core-3.0-ia32:core-3.0-noarch:core-3.1-ia32:core-3.1-noarch:core-3.2-ia32:core-3.2-noarch:core-4.0-ia32:core-4.0-noarch
Distributor ID:	Ubuntu
Description:	Ubuntu 12.04.4 LTS
Release:	12.04
Codename:	precise

$ dpkg -l libtiff* | grep ii
ii  libtiff-tools                          3.9.5-2ubuntu1.5                         TIFF manipulation and conversion tools
ii  libtiff4                               3.9.5-2ubuntu1.5                         Tag Image File Format (TIFF) library
ii  libtiff4-dev                           3.9.5-2ubuntu1.5                         Tag Image File Format library (TIFF), development files
ii  libtiffxx0c2                           3.9.5-2ubuntu1.5                         Tag Image File Format (TIFF) library -- C++ interface

Alan as well encounter this issue, more details in the related ​rasdaman-dev thread.

An other useful link: ​http://www.imagemagick.org/discourse-server/viewtopic.php?f=1&t=20533

[abeccati]

If this resolves to be a defect in the tiff libraries which is solved by a subsequent release of it then we should prominently mark this incompatibility (in the download or install pages or any better location that we can devise for that purpose).

It seems that this works:

libtiff4 3.9.5-2ubuntu1.5

While this doesn't

libtiff4 3.9.6-11 +
libtiff5 4.0.2-6

Do we have a one liner query that users can use to test this and report us about their versions? We should prepare it and announce a request for feedback on the users list imho.
Indeed being unable to load tiffs is quite a showstopper, users might be able to provide workaround instructions to others if we discuss this on the users list.

[Dimitar Misev]

Comment from Piero:

Dimitar has no problem with it and has:

libtiff4 3.9.6-11 +
libtiff5 4.0.2-6

in his laptop, I have:

libtiff4 3.9.5-2ubuntu1.5

[Dimitar Misev]

So Alan I suppose you've mixed what works and what doesn't in your comment?

[Dimitar Misev]

I believe this is some issue of library incompatibilities, as in the imagemagick thread, rasdaman is built with one libtiff but the uses another in inv_tiff.

[Dimitar Misev]

Piero can you also post which GDAL you have? I have

$ gdalinfo --version
GDAL 1.9.0, released 2011/12/29

[Dimitar Misev]

From what I could figure out this is a clash between GDAL and libtiff.
It seems that GDAL embeds and patches libtiff, which has that assert: ​http://trac.osgeo.org/gdal/browser/tags/1.9.0/gdal/frmts/gtiff/libtiff/tif_open.c?rev=23700
libtiff on the other hand doesn't have it anywhere in its source.

[Piero Campalani]

Same here.

Replying to dmisev:

Piero can you also post which GDAL you have? I have

$ gdalinfo --version
GDAL 1.9.0, released 2011/12/29

[Dimitar Misev]

Most probably the issue is because the GDAL you have has been compiled with the asserts enabled, which should not be done for released version but only in development.

[Dimitar Misev]

Piero, there is a decode function in rasdaman now, can you test with it? (just substitute inv_tiff with decode)

[Piero Campalani]

HI,
I tried (latest-dev changeset:8f46ab0) with the decode function but still I cannot import a TIF file:

> RASQL='rasql --user rasadmin --passwd rasadmin'
> $RASQL -q "create collection test_inv_tiff GreySet"
  [OK]
> $RASQL -q "insert into test_inv_tiff values (char) decode(\$1)" -f /path/to/mean_summer_airtemp.tif
    rasql: rasdaman query tool v1.0, rasdaman v9.0.0-g8f46ab0 -- generated on 05.03.2014 14:39:04.
    opening database RASBASE at localhost:7001...ok
    fetching type information for GreyString from database, using readonly transaction...ok
    reading file systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif...ok
      constant 1: GMarray
      Oid...................: 
      Type Structure........: 
      Type Schema...........: marray< char >
      Domain................: [0:631211]
      Base Type Schema......: char
      Base Type Length......: 1
      Data format.......... : Array
      Data size (bytes).... : 631212
    Executing insert query...rasdaman error 0: Exception: Transfer Failed
    aborting transaction...ok
    rasql done.

nohup log says:

[bt]: (2) /home/rasdaman/install/bin/rasserver (mddbasetypecommon.cc:131) - MDDBaseType::getBaseType() const+0x6 [0x82862f6]
[bt]: (3) /home/rasdaman/install/bin/rasserver (qtunaryinduce.cc:843) - QtCast::checkType(QtTypeTuple*)+0x7a [0x81c3c70]
[bt]: (4) /home/rasdaman/install/bin/rasserver (qtinsert.cc:488) - QtInsert::checkType()+0x36 [0x81dcda8]
[bt]: (5) /home/rasdaman/install/bin/rasserver (querytree.cc:120) - QueryTree::checkSemantics()+0x72 [0x81f09e0]
[bt]: (6) /home/rasdaman/install/bin/rasserver (servercomm2.cc:2268) - ServerComm::executeInsert(unsigned long, char const*, ExecuteQueryRes&)+0x220 [0x8178a68]
[bt]: (7) /home/rasdaman/install/bin/rasserver (rasserver_entry.cc:307) - RasServerEntry::compat_ExecuteInsertQuery(char const*, ExecuteQueryRes&)+0x29 [0x816d69f]
[bt]: (8) /home/rasdaman/install/bin/rasserver (rnpservercomm.cc:769) - RnpRasDaManComm::executeInsertQuery()+0x5b [0x8188665]
[bt]: (9) /home/rasdaman/install/bin/rasserver (rnpservercomm.cc:361) - RnpRasDaManComm::decodeFragment()+0x2a5 [0x818783d]
[bt]: (10) /home/rasdaman/install/bin/rasserver (rnpservercomm.cc:132) - RnpRasDaManComm::processRequest(akg::CommBuffer*, akg::CommBuffer*, rnp::RnpTransport::CarrierProtocol, rnp::RnpServerJob*)+0x194 [0x81871e0]
[bt]: (11) /home/rasdaman/install/bin/rasserver (rnpcommunication.cc:415) - rnp::RnpServerJob::processRequest()+0x4e [0x818dae2]
[bt]: (12) /home/rasdaman/install/bin/rasserver (akgnet_nbcomm.cc:483) - akg::NbCommunicator::processJobs()+0x61 [0x82abc53]
[bt]: (13) /home/rasdaman/install/bin/rasserver (akgnet_nbcomm.cc:453) - akg::NbCommunicator::mainLoop()+0xa3 [0x82abb99]
[bt]: (14) /home/rasdaman/install/bin/rasserver (akgnet_nbcomm.cc:422) - akg::NbCommunicator::runServer()+0x4d [0x82abadf]
[bt]: (15) /home/rasdaman/install/bin/rasserver (rnpserver.cc:99) - startRnpServer()+0x2ac [0x8186564]
Spawned rasserver v9.0.0-g8f46ab0 on base DBMS pgsql -- generated on 05.03.2014 14:39:04.

NOTE: as a workaround, importing TIFFs with [RasgeoUserGuide rasgeo]/rasimport works fine.

[Dimitar Misev]

George, can you look at the error above in comment:10? The test file is in

./applications/petascope/src/main/db/petascope/mean_summer_airtemp.tif
./systemtest/testcases_services/test_wcps/testdata/mean_summer_airtemp.tif

[George Merticariu]

The problem will be fixed once the Ticket#777 will be merged.

[Dimitar Misev]

I think we can't do much about this, so closing.