SMAC-M - Scripts for Map And Chart Manager

This project allows you to convert nautical dataset and display it with Mapserver. It include all utilities to convert data from S57 to a shapefiles and to create mapfiles for a Mapserver WMS service. This package uses the a python script and bash scripts to build a complete mapfiles from a set of templates and styling information to enable display of nautical data in the format S57.

greenhead-caostal wingaersheekbeach-harbour

The package enables mapserver to display nautical charts in different color modes and with alternative color tables. Currently only simplified S52 symbology is carried out. But future plans are to include paper chart symbology as a separate style. The package also contains tools to create AML mapfiles for Mapserver.

The data handling is carried out with the OGR package. For performance reasons all S57 data is converted into shape format and every object type is split into separate files. Separate files are also created for point, line and polygon features

Furhter on a tool is provided to process tiff files and create mapfiles automaticly.

Requirement

sudo apt install python3 python3-pip python3-venv gdal-bin python3-gdal build-essential libgdal-dev imagemagick xmlstarlet
echo 'PATH="$HOME/.local/bin/:$PATH"' >>~/.bashrc
pip3 install --user pipenv

Clone SMAC-M and install Python virtual environment

A good way to run and test this tool is to create a development virtual environment available only for the active user. Before running the scripts for the first time, you must install their dependencies. This project uses Pipenv to manage its virtual environment.

After downloading a new version of the code, run pipenv install in the root directory to install the bulk of the dependencies.

Due to the way GDAL works, it must be installed separately with the version that matches the version of GDAL installed on the system. To install GDAL in the virtual environment, run pipenv run pip install "GDAL<=$(gdal-config --version)"

Once the packages have been installed, use pipenv shell to activate the environment.

git clone https://github.com/LarsSchy/SMAC-M
cd ./SMAC-M
pipenv install
pipenv run pip install "GDAL<=$(gdal-config --version)"
pipenv shell

Up to date GDAL/OGR

GDAL/OGR used OpenCPN configuration file to read and extract data from S-57 dataset. If you don't have up to date metadata file on your system, you will be unable to read all the data available in S-57 dataset. You have to make sure to update those files on your system to make run properly data converter script. In S-57 OGR driver documentation said: _The S-57 reader depends on having two supporting files, s57objectclasses.csv, and s57attributes.csv available at runtime in order to translate features in an object class specific manner. These should be in the directory pointed to by the environment variable S57CSV, or in the current working directory

For your setup and future usage, be sure to find GDAL/OGR S-57 configuration file, and update them like this:

$ ogrinfo --version
GDAL 2.2.2, released ...
$ ls /usr/share/gdal/
2.2

Create up to date s57 object files in GDAL data folder /usr/share/gdal/2.2/

$ wget https://raw.githubusercontent.com/OpenCPN/OpenCPN/master/data/s57data/s57objectclasses.csv
$ wget https://raw.githubusercontent.com/OpenCPN/OpenCPN/master/data/s57data/s57attributes.csv
$ cp s57objectclasses.csv /usr/share/gdal/2.2/s57objectclasses_iw.csv
$ cp s57attributes.csv /usr/share/gdal/2.2/s57attributes_iw.csv

Add extra OGR environment variable on your system

export S57_PROFILE

To process sound depth points properly and other Multipoint dataset, you have to set OGR environment variable on your system

export OGR_S57_OPTIONS=SPLIT_MULTIPOINT=ON,ADD_SOUNDG_DEPTH=ON

You can test your setup by using ogrinfo and other ogr2ogr tools with S57 profile like this:

$ ogrinfo --config S57_PROFILE iw -ro ./datatest/US5OH10M/US5OH10M.000 | grep "SLOTOP"
48: SLOTOP

Generating the mapfile

Generate a configuration file

This configuration file contains the options for a particular map generation. It is in TOML format.

You can easily generate a configuration file with the generate_toml_config.py script. This script accepts the same options as the legacy generate_map_config.py and creates a toml file with these options.

 python3 bin/generate_toml_config.py \
     --chart ./shp_s57data/ENC_ROOT \
     -rule-default-color IHO \
     --chartsymbols ./chartsymbols_S57.xml \
     -enhancedchartdata ./shp_s57data/shp \
     --tablename Paper \
     --displaycategory Standard \
     --rule-set-path ../resources/rules/ \
     -o configfile.toml

If you prefer to create your configuration file manually, you can use config.sample.toml as a base.

For the rest of these instructions, the configuration file will be referred to as configfile.toml

Convert S-57 dataset

This script will build all data needed for nautical charts map service based on OpenCPN configuration file and support light sectors layers.

The script will create shapefiles for S-57 objects ONLY if data is found in your source files.

python3 bin/generate_shapefiles.py configfile.toml

This same script can be used whenever the S-57 dataset is updated to regenerate the shapefiles.

Light sector

This project will automatically create lights sector shapefiles based on LIGHTS dataset when generating the shapefiles

capsmall-lightsector

In case you need to update Lights sector or change the default radius, you can simply run this script:

cd chart-installation/data_files_conversion/shp_s57data
python3 generate_light_sector.py [input_lights_shp_path] [radius]
  # NOTE 1: input shapefile must be named as *_LIGHTS_*.shp
  # NOTE 2: if radius = valnmr keyword, distance will be take from data

Generating mapfiles

The mapfiles can be generated from the information in the configuration file.

python3 chart-installation/generate_map_files/generate_map_config.py configfile.toml

Manage scale of navigation level

By default, Maximum scale visibility can be adjust by editing this file: ./chart-installation/generate_map_files/resources/rules/layer_rules/layer_msd.csv

|Navigation Level| Max Scale| |1|10000000| |2|5000000| |3|1000000| |4|500000| |5|100000| |6|50000|

Be award if you change default maxscale visibility, you may need to adjust sounding points max scale shift in your configuration variable file (config.toml)

sounding_maxscale_shift = 0.4

Enhanced data mapfile limitations

Working with enhanced data allows to create mapfiles from the chartsymbols.xml file. This file contains all the specification of all symbols of the IENC symbology and is provided by OpenCPN. The file provided by OpenCPN does contains a few errors or limitation that are not currently handled.

Generating Symbolset

The symbols are automatically generated from the OpenCPN symbols when the mapfiles are updated. They do not need to be generated manually

Testing

Follow those steps to get mapserver up and running. https://github.com/mapserver/mapserver/wiki/Install-mapserver-on-ubuntu-18.04-from-official-repository

Then You should be able to test the configuration in the built in open layers viewer with:

http://localhost/cgi-bin/mapserv?map=/data/Chart_dir/map/SeaChart_DAY_BRIGHT.map&SERVICE=WMS&REQUEST=Getmap&VERSION=1.1.1&LAYERS=SeaChart_DAY_BRIGHT&srs=EPSG:3006&BBOX=133870,5798110,1541520,7459340&FORMAT=application/openlayers&WIDTH=2000&HEIGHT=1100

NOTE: You should adjust to EPSG:3857 and a suitable BBOX for your data.

NOAA ENC service

The commands used to generate a basic NOAA ENC map service have been moved to their separate readme.

General information

IENC (S-57) data is compiled for a variety of navigational purposes. This project is built to mainely support all of them. First version will support the first 6 navigation purpose. At the end, all of thos levels will be converte to mapfiles.

The following directory structure is mandatory. On each volume within an exchange set there must be a root directory called ENC_ROOT. Data is publish throught .000 source file and data update are store into .001, .002 and so on. OGR2OGR software is able to read all this data.

NL600021.000
NL600021.001
NL600021.002

A valid data set file must be uniquely identified world wide by its name. The data set files are named according to the specifications given below:

 CCPRRRRR.EEE
 | | |    |
 | | |    |------------ EEE = update number
 | | |
 | | |----------------- RRRRR = waterway code and waterway distance (kilometre) or
 | |                            identification of the equivalent paper chart number
 | |
 | |------------------- P = navigational purpose
 | 
 |--------------------- CC = producer code