MS RFC 68: Support for combining features from multiple layers

Date:2011/02/10 Author:Tamas Szekeres Contact:szekerest at gmail.com Last Edited:$Date$ Status:Adopted (Implemented on 2011-03-04) Version:MapServer 6.0 Id:$Id$

Description: This RFC proposes an implementation for creating a new data provider (CONNECTIONTYPE=UNION) which provides an option to represent features from multiple layers in a single layer.

1. Overview

Today, you can combine multiple files using a tileindex, but only if they have the same attributes, and description. In theory, it might be nice to be able to have multiple source layers that have similar attributes and to be able to combine them into a single layer (being called as union layer) using some compatible subset of the source column attributes so that the combined layer could be treated as a single layer. We could then use this single layer in the same way as any other layer when setting up the classes styles and labels. This layer could also provide the option to use features from multiple layers when configuring clustering support as per MS RFC 69.

2. The proposed solution

This functionality will be implemented as a separate layer data provider (CONNECTIONTYPE=UNION). The union layer will use the CONNECTION parameter to define the source layers in the mapfiles as follows:

LAYER
  CONNECTIONTYPE UNION
  CONNECTION "layer1,layer2,layer3" # reference to the source layers
  NAME "union-layer"
  PROCESSING "ITEMS=itemname1,itemname2,itemname3"
  ...
END
LAYER
  CONNECTIONTYPE OGR
  NAME layer1
  ...
END
LAYER
  CONNECTIONTYPE SHAPE
  NAME layer2
  ...
END
LAYER
  CONNECTIONTYPE SHAPE
  NAME layer3
  ...
END

The new layer data provider will work in the following way:

1. In LayerOpen, it will open all of the source layers specified by the CONNECTION parameter
2. The LayerWhichShapes call will be delegated to the underlying layers and provide the rectangle for the area of interest
3. LayerNextShape will iterate through the layers and call LayerNextShape for the subsequent shapes. The layer index is assigned to the tileindex of the returned shapes. If we finish retrieving the shapes from one layer we start retrieving the features from the next source layer.
4. LayerGetShape will identify the layer based on the tile index and then call LayerGetShape of this layer

The union layer and the source layers must have the same geometry type otherwise an error is generated in the LayerOpen call.

2.1 Handling the layer attributes (items)

In general the source layers must provide those attributes which are required when rendering the union layer, however the underlying data may contain further attributes, which are not used when fetching the data from the original source. When all attributes are requested (in the query operations) then the union layer will provide only some aggegated attributes (like the layer name or the group name of the source layer the feature belongs to). The set of the items can manually be overridden (and further attributes can be exposed) by using the existing ITEMS processing option. At this stage of the development, the driver will expose the following additional attributes:

1. Combine:SourceLayerName - The name of the source layer the feature belongs to
2. Combine:SourceLayerGroup - The group of the source layer the feature belongs to

2.2 Projections

It is suggested to use the same projection of the union layer and the source layers. The layer provider will however support transforming the feature positions between the source layers and the union layer.

2.3 Handling classes and styles

We can define the symbology and labelling of the union layers in the same way as any other layer by specifying the classes and styles. In addition we will also support the STYLEITEM AUTO option for the union layer, which is essential if we want to display the features in the same way as with the source layers. The source layers may also use the STYLEITEM AUTO setting if the underlying data source provides that.

2.4 Query processing

The driver will create (and operate on) a copy of the source layers which will allow to keep the underlying layers open until the union layer is open. This will provide the single pass query to work in case if the source layer supports it. The queries on the union layer will anyway behave the same like with the other layers.

3. Implementation Details

In order to implement this enhancement the following changes should be made in the MapServer codebase:

1. Modify the lexer to interpret the new connection type (UNION).
2. Implement mapunion.c containing the code of the union layer data source.

3.1 Files affected

The following files will be modified/created by this RFC:

maplexer.l (maplexer.c)
mapserver.h
Makefile.vc
Makefile.in
mapunion.c (new)

3.2 MapScript Issues

The connection type of this new layer will be exposed in the MapScript interface. No further impact is taken into account at the moment.

3.3 Backwards Compatibilty Issues

This change provides a new functionality with no backwards compatibility issues being considered.

4. Bug ID

The ticket for RFC-68 (containing implementation code) can be found here.

Bug 3674

5. Voting history

+1 from SteveW, DanielM, AssefaY, PericlesN, ThomasB, SteveL, TamasS