Skip to content

Commit

Permalink
Merge pull request #13 from loicgasser/from_string_io
Browse files Browse the repository at this point in the history 
Add fromStringIO
  • Loading branch information
@loicgasser
loicgasser authored Nov 18, 2016
2 parents 9826da8 + 2f7b418 commit 882b41f
Show file tree
Hide file tree
Showing 5 changed files with 176 additions and 101 deletions.
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
*.pyc
*.egg-info
.venv
venv
*.swp
dist/
doc/build/
2 changes: 1 addition & 1 deletion dev-requirements.txt
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
nose
flake8
coveralls
mock
mock==1.0.1
sphinx
sphinx_rtd_theme
sphinx-autobuild
47 changes: 36 additions & 11 deletions doc/source/tutorial.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Introduction
------------

This tutorial shows how to use the quantized mesh tile module.
This format is designed to work exclusively with the `TMS (Tile Map Service)`_ tiling scheme
This format is designed to work exclusively with the `TMS (Tile Map Service)`_ tiling scheme
and is optimized to display `TIN (Triangulated irregular network)`_ data on the web.

.. _TMS (Tile Map Service):
Expand All @@ -25,7 +25,7 @@ This module has been developed based on the specifications of the format describ
.. _here:
http://cesiumjs.org/data-and-assets/terrain/formats/quantized-mesh-1.0.html

Therefore, if you've planned on creating your own terrain server, please make sure you follow all the instructions
Therefore, if you've planned on creating your own terrain server, please make sure you follow all the instructions
provided in the specifications of the format. You may also need to define a `layer.json`_ metadata file at the root of your
terrain server which seems to be a derivative of `Mapbox tilejson-spec`_. Make sure you get it right. ;)

Expand All @@ -38,7 +38,7 @@ Create a terrain tile
---------------------

The encoding module can determine the extent of a tile based on the geometries it receives as arguments.
Nevertheless, this can only work if the tile has at least 1 triangle on all of its 4 edges.
Nevertheless, this can only work if the tile has at least 1 triangle on all of its 4 edges.
As a result, it is advised to always provide the bounds of the tile.

So first determine the extent of the tile.
Expand All @@ -48,7 +48,7 @@ So first determine the extent of the tile.
>>> [z, x, y] = [9, 533, 383]
>>> [west, south, east, north] = bounds = geodetic.TileBounds(x, y, z)

Now that we have the extent, let's assume you generated a mesh using your favorite meshing engine and ended up with
Now that we have the extent, let's assume you generated a mesh using your favorite meshing engine and ended up with
the following geometries.

>>> geometries = [
Expand All @@ -68,7 +68,7 @@ the following geometries.
'7.5585937 44.82421875 310.2, ' +
'7.3828125 45.0 320.2, ' +
'7.734375 45.0 330.3))'
]
]

For a full list of input formats for the geometries, please refer to :class:`quantized_mesh_tile.terrain.TerrainTile`.

Expand All @@ -91,7 +91,7 @@ To define a water-mask you can use:
>>> watermask = [255]
>>> tile = encode(geometries, bounds=bounds, watermask=watermask)

If you have non-triangular geometries (typically when clipping in an existing mesh),
If you have non-triangular geometries (typically when clipping in an existing mesh),
you can also use the option `autocorrectGeometries` to collapse them into triangles.
This option should be used with care to fix punctual meshing mistakes.

Expand All @@ -113,25 +113,50 @@ This option should be used with care to fix punctual meshing mistakes.
'7.3828125 45.0 320.2, ' +
'7.55859375 45.0 325.2, ' +
'7.734375 45.0 330.3))'
]
]
>>> tile = encode(geometries, bounds=bounds, autocorrectGeometries=True)
>>> print len(tile.getTrianglesCoordinates())


Read a terrain tile
-------------------
Read a local terrain tile
-------------------------

As the coordinates within a terrain tile are `quantized`, its values can be only be correctly decoded if we know the exact
As the coordinates within a terrain tile are `quantized`, its values can be only be correctly decoded if we know the exact
extent of the tile.

>>> from quantized_mesh_tile.global_geodetic import GlobalGeodetic
>>> geodetic = GlobalGeodetic(True)
>>> [z, x, y] = [9, 533, 383]
>>> [west, south, east, north] = bounds = geodetic.TileBounds(x, y, z)
>>> path = '%s/%s/%s.terrain' % (z, x, y)'
>>> path = '%s/%s/%s.terrain' % (z, x, y)
>>> tile = decode(path, bounds)
>>> print tile.getTrianglesCoordinates()

Or let's assume we have a gizpped compressed tile with water-mask extension:

>>> tile = decode(path, bounds, gzipped=True, hasWatermask=True)

Read a remote terrain tile
--------------------------

Using the `requests module`_, here is an example on how to read a remote terrain tile.

.. _requests module:
http://docs.python-requests.org/en/master/

The you won't need to decompress the gzipped tile has this is performed automatically
in the requests module.

>>> import cStringIO
>>> import requests
>>> from quantized_mesh_tile.terrain import TerrainTile
>>> from quantized_mesh_tile.global_geodetic import GlobalGeodetic
>>> [z, x, y] = [14, 24297, 10735]
>>> geodetic = GlobalGeodetic(True)
>>> [west, south, east, north] = bounds = geodetic.TileBounds(x, y, z)
>>> url = 'http://assets.agi.com/stk-terrain/world/%s/%s/%s.terrain?v=1.16389.0' % (z, x, y)
>>> response = requests.get(url)
>>> content = cStringIO.StringIO(response.content)
>>> ter = TerrainTile(west=west, south=south, east=east, north=north)
>>> ter.fromStringIO(content)
>>> print ter.getVerticesCoordinates()
197 changes: 108 additions & 89 deletions quantized_mesh_tile/terrain.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,112 @@ def _computeVerticesCoordinates(self):
)
)

def fromStringIO(self, f, hasLighting=False, hasWatermask=False):
"""
A method to read a terrain tile content.
Arguments:
``f``
An instance of cStringIO.StingIO containing the terrain data. (Required)
``hasLighting``
Indicate if the tile contains lighting information. Default is ``False``.
``hasWatermask``
Indicate if the tile contains watermask information. Default is ``False``.
"""
self.hasLighting = hasLighting
self.hasWatermask = hasWatermask
# Header
for k, v in TerrainTile.quantizedMeshHeader.iteritems():
self.header[k] = unpackEntry(f, v)

# Delta decoding
ud = 0
vd = 0
hd = 0
# Vertices
vertexCount = unpackEntry(f, TerrainTile.vertexData['vertexCount'])
for i in xrange(0, vertexCount):
ud += zigZagDecode(
unpackEntry(f, TerrainTile.vertexData['uVertexCount'])
)
self.u.append(ud)
for i in xrange(0, vertexCount):
vd += zigZagDecode(
unpackEntry(f, TerrainTile.vertexData['vVertexCount'])
)
self.v.append(vd)
for i in xrange(0, vertexCount):
hd += zigZagDecode(
unpackEntry(f, TerrainTile.vertexData['heightVertexCount'])
)
self.h.append(hd)

# Indices
meta = TerrainTile.indexData16
if vertexCount > TerrainTile.BYTESPLIT:
meta = TerrainTile.indexData32
triangleCount = unpackEntry(f, meta['triangleCount'])
ind = unpackIndices(f, triangleCount * 3, meta['indices'])
self.indices = decodeIndices(ind)

meta = TerrainTile.EdgeIndices16
if vertexCount > TerrainTile.BYTESPLIT:
meta = TerrainTile.indexData32
# Edges (vertices on the edge of the tile)
# Indices (are the also high water mark encoded?)
westIndicesCount = unpackEntry(f, meta['westVertexCount'])
self.westI = unpackIndices(f, westIndicesCount, meta['westIndices'])

southIndicesCount = unpackEntry(f, meta['southVertexCount'])
self.southI = unpackIndices(f, southIndicesCount, meta['southIndices'])

eastIndicesCount = unpackEntry(f, meta['eastVertexCount'])
self.eastI = unpackIndices(f, eastIndicesCount, meta['eastIndices'])

northIndicesCount = unpackEntry(f, meta['northVertexCount'])
self.northI = unpackIndices(f, northIndicesCount, meta['northIndices'])

if self.hasLighting:
# One byte of padding
# Light extension header
meta = TerrainTile.ExtensionHeader
extensionId = unpackEntry(f, meta['extensionId'])
if extensionId == 1:
extensionLength = unpackEntry(f, meta['extensionLength'])

# Consider padding of 2 bits
# http://cesiumjs.org/data-and-assets/terrain/formats/quantized-mesh-1.0.html
f.read(2)

for i in xrange(0, (extensionLength / 2) - 1):
x = unpackEntry(f, TerrainTile.OctEncodedVertexNormals['xy'])
y = unpackEntry(f, TerrainTile.OctEncodedVertexNormals['xy'])
self.vLight.append(octDecode(x, y))

if self.hasWatermask:
meta = TerrainTile.ExtensionHeader
extensionId = unpackEntry(f, meta['extensionId'])
if extensionId == 2:
extensionLength = unpackEntry(f, meta['extensionLength'])
row = []
for i in xrange(0, extensionLength):
row.append(unpackEntry(f, TerrainTile.WaterMask['xy']))
if len(row) == 256:
self.watermask.append(row)
row = []
if len(row) > 0:
self.watermask.append(row)

data = f.read(1)
if data:
raise Exception('Should have reached end of file, but didn\'t')

def fromFile(self, filePath, hasLighting=False, hasWatermask=False, gzipped=False):
"""
A method to read a terrain tile file. It is assumed that the tile unzipped.
Expand All @@ -315,103 +421,16 @@ def fromFile(self, filePath, hasLighting=False, hasWatermask=False, gzipped=Fals
``hasWatermask``
Indicate if the tile contains watermask information. Default is ``True``.
Indicate if the tile contains watermask information. Default is ``False``.
``gzipped``
Indicate if the tile content is gzipped. Default is ``False``.
"""
self.hasLighting = hasLighting
self.hasWatermask = hasWatermask
with open(filePath, 'rb') as f:
if gzipped:
f = ungzipFileObject(f)

# Header
for k, v in TerrainTile.quantizedMeshHeader.iteritems():
self.header[k] = unpackEntry(f, v)

# Delta decoding
ud = 0
vd = 0
hd = 0
# Vertices
vertexCount = unpackEntry(f, TerrainTile.vertexData['vertexCount'])
for i in xrange(0, vertexCount):
ud += zigZagDecode(
unpackEntry(f, TerrainTile.vertexData['uVertexCount'])
)
self.u.append(ud)
for i in xrange(0, vertexCount):
vd += zigZagDecode(
unpackEntry(f, TerrainTile.vertexData['vVertexCount'])
)
self.v.append(vd)
for i in xrange(0, vertexCount):
hd += zigZagDecode(
unpackEntry(f, TerrainTile.vertexData['heightVertexCount'])
)
self.h.append(hd)

# Indices
meta = TerrainTile.indexData16
if vertexCount > TerrainTile.BYTESPLIT:
meta = TerrainTile.indexData32
triangleCount = unpackEntry(f, meta['triangleCount'])
ind = unpackIndices(f, triangleCount * 3, meta['indices'])
self.indices = decodeIndices(ind)

meta = TerrainTile.EdgeIndices16
if vertexCount > TerrainTile.BYTESPLIT:
meta = TerrainTile.indexData32
# Edges (vertices on the edge of the tile)
# Indices (are the also high water mark encoded?)
westIndicesCount = unpackEntry(f, meta['westVertexCount'])
self.westI = unpackIndices(f, westIndicesCount, meta['westIndices'])

southIndicesCount = unpackEntry(f, meta['southVertexCount'])
self.southI = unpackIndices(f, southIndicesCount, meta['southIndices'])

eastIndicesCount = unpackEntry(f, meta['eastVertexCount'])
self.eastI = unpackIndices(f, eastIndicesCount, meta['eastIndices'])

northIndicesCount = unpackEntry(f, meta['northVertexCount'])
self.northI = unpackIndices(f, northIndicesCount, meta['northIndices'])

if self.hasLighting:
# One byte of padding
# Light extension header
meta = TerrainTile.ExtensionHeader
extensionId = unpackEntry(f, meta['extensionId'])
if extensionId == 1:
extensionLength = unpackEntry(f, meta['extensionLength'])

# Consider padding of 2 bits
# http://cesiumjs.org/data-and-assets/terrain/formats/quantized-mesh-1.0.html
f.read(2)

for i in xrange(0, (extensionLength / 2) - 1):
x = unpackEntry(f, TerrainTile.OctEncodedVertexNormals['xy'])
y = unpackEntry(f, TerrainTile.OctEncodedVertexNormals['xy'])
self.vLight.append(octDecode(x, y))

if self.hasWatermask:
meta = TerrainTile.ExtensionHeader
extensionId = unpackEntry(f, meta['extensionId'])
if extensionId == 2:
extensionLength = unpackEntry(f, meta['extensionLength'])
row = []
for i in xrange(0, extensionLength):
row.append(unpackEntry(f, TerrainTile.WaterMask['xy']))
if len(row) == 256:
self.watermask.append(row)
row = []
if len(row) > 0:
self.watermask.append(row)

data = f.read(1)
if data:
raise Exception('Should have reached end of file, but didn\'t')
self.fromStringIO(f, hasLighting=hasLighting, hasWatermask=hasWatermask)

def toStringIO(self, gzipped=False):
"""
Expand Down
30 changes: 30 additions & 0 deletions tests/test_terrain_tile.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -254,3 +254,33 @@ def testGzippedTileCreationFromTopology(self):

fileLike = tile.toStringIO(gzipped=True)
self.assertIsInstance(fileLike, cStringIO.OutputType)

def testFromStringIO(self):
z = 10
x = 1563
y = 590
geodetic = GlobalGeodetic(True)
[minx, miny, maxx, maxy] = geodetic.TileBounds(x, y, z)

# Regular file not gzip compressed
ter = TerrainTile()
ter = TerrainTile(west=minx, south=miny, east=maxx, north=maxy)
with open('tests/data/%s_%s_%s_light_watermask.terrain' % (z, x, y)) as f:
content = cStringIO.StringIO(f.read())

ter.fromStringIO(content, hasLighting=True, hasWatermask=True)

# check indices
self.assertGreater(len(ter.indices), 0)

# check edges
self.assertGreater(len(ter.westI), 0)
self.assertGreater(len(ter.southI), 0)
self.assertGreater(len(ter.eastI), 0)
self.assertGreater(len(ter.northI), 0)

# check extensions
self.assertEqual(len(ter.watermask), 1)
self.assertEqual(len(ter.watermask[0]), 1)
# Water only -> 255
self.assertEqual(ter.watermask[0][0], 255)

0 comments on commit 882b41f

Please sign in to comment.