Table of Contents for
Practical GIS

Version ebook / Retour

Cover image for bash Cookbook, 2nd Edition Practical GIS by Gábor Farkas Published by Packt Publishing, 2017
  1. Practical GIS
  2. Title Page
  3. Copyright
  4. Credits
  5. About the Author
  6. About the Reviewer
  7. www.PacktPub.com
  8. Customer Feedback
  9. Dedication
  10. Table of Contents
  11. Preface
  12. What this book covers
  13. What you need for this book
  14. Who this book is for
  15. Conventions
  16. Reader feedback
  17. Customer support
  18. Downloading the example code
  19. Downloading the color images of this book
  20. Errata
  21. Piracy
  22. Questions
  23. Setting Up Your Environment
  24. Understanding GIS
  25. Setting up the tools
  26. Installing on Linux
  27. Installing on Windows
  28. Installing on macOS
  29. Getting familiar with the software
  30. About the software licenses
  31. Collecting some data
  32. Getting basic data
  33. Licenses
  34. Accessing satellite data
  35. Active remote sensing
  36. Passive remote sensing
  37. Licenses
  38. Using OpenStreetMap
  39. OpenStreetMap license
  40. Summary
  41. Accessing GIS Data With QGIS
  42. Accessing raster data
  43. Raster data model
  44. Rasters are boring
  45. Accessing vector data
  46. Vector data model
  47. Vector topology - the right way
  48. Opening tabular layers
  49. Understanding map scales
  50. Summary
  51. Using Vector Data Effectively
  52. Using the attribute table
  53. SQL in GIS
  54. Selecting features in QGIS
  55. Preparing our data
  56. Writing basic queries
  57. Filtering layers
  58. Spatial querying
  59. Writing advanced queries
  60. Modifying the attribute table
  61. Removing columns
  62. Joining tables
  63. Spatial joins
  64. Adding attribute data
  65. Understanding data providers
  66. Summary
  67. Creating Digital Maps
  68. Styling our data
  69. Styling raster data
  70. Styling vector data
  71. Mapping with categories
  72. Graduated mapping
  73. Understanding projections
  74. Plate Carrée - a simple example
  75. Going local with NAD83 / Conus Albers
  76. Choosing the right projection
  77. Preparing a map
  78. Rule-based styling
  79. Adding labels
  80. Creating additional thematics
  81. Creating a map
  82. Adding cartographic elements
  83. Summary
  84. Exporting Your Data
  85. Creating a printable map
  86. Clipping features
  87. Creating a background
  88. Removing dangling segments
  89. Exporting the map
  90. A good way for post-processing - SVG
  91. Sharing raw data
  92. Vector data exchange formats
  93. Shapefile
  94. WKT and WKB
  95. Markup languages
  96. GeoJSON
  97. Raster data exchange formats
  98. GeoTIFF
  99. Clipping rasters
  100. Other raster formats
  101. Summary
  102. Feeding a PostGIS Database
  103. A brief overview of databases
  104. Relational databases
  105. NoSQL databases
  106. Spatial databases
  107. Importing layers into PostGIS
  108. Importing vector data
  109. Spatial indexing
  110. Importing raster data
  111. Visualizing PostGIS layers in QGIS
  112. Basic PostGIS queries
  113. Summary
  114. A PostGIS Overview
  115. Customizing the database
  116. Securing our database
  117. Constraining tables
  118. Saving queries
  119. Optimizing queries
  120. Backing up our data
  121. Creating static backups
  122. Continuous archiving
  123. Summary
  124. Spatial Analysis in QGIS
  125. Preparing the workspace
  126. Laying down the rules
  127. Vector analysis
  128. Proximity analysis
  129. Understanding the overlay tools
  130. Towards some neighborhood analysis
  131. Building your models
  132. Using digital elevation models
  133. Filtering based on aspect
  134. Calculating walking times
  135. Summary
  136. Spatial Analysis on Steroids - Using PostGIS
  137. Delimiting quiet houses
  138. Proximity analysis in PostGIS
  139. Precision problems of buffering
  140. Querying distances effectively
  141. Saving the results
  142. Matching the rest of the criteria
  143. Counting nearby points
  144. Querying rasters
  145. Summary
  146. A Typical GIS Problem
  147. Outlining the problem
  148. Raster analysis
  149. Multi-criteria evaluation
  150. Creating the constraint mask
  151. Using fuzzy techniques in GIS
  152. Proximity analysis with rasters
  153. Fuzzifying crisp data
  154. Aggregating the results
  155. Calculating statistics
  156. Vectorizing suitable areas
  157. Using zonal statistics
  158. Accessing vector statistics
  159. Creating an atlas
  160. Summary
  161. Showcasing Your Data
  162. Spatial data on the web
  163. Understanding the basics of the web
  164. Spatial servers
  165. Using QGIS for publishing
  166. Using GeoServer
  167. General configuration
  168. GeoServer architecture
  169. Adding spatial data
  170. Tiling your maps
  171. Summary
  172. Styling Your Data in GeoServer
  173. Managing styles
  174. Writing SLD styles
  175. Styling vector layers
  176. Styling waters
  177. Styling polygons
  178. Creating labels
  179. Styling raster layers
  180. Using CSS in GeoServer
  181. Styling layers with CSS
  182. Creating complex styles
  183. Styling raster layers
  184. Summary
  185. Creating a Web Map
  186. Understanding the client side of the Web
  187. Creating a web page
  188. Writing HTML code
  189. Styling the elements
  190. Scripting your web page
  191. Creating web maps with Leaflet
  192. Creating a simple map
  193. Compositing layers
  194. Working with Leaflet plugins
  195. Loading raw vector data
  196. Styling vectors in Leaflet
  197. Annotating attributes with popups
  198. Using other projections
  199. Summary
  200. Appendix

Securing our database

So far, we used the public schema with the postgres role to create and fill our database. These are the default values, which are great for creating a local database, but far from ideal if we would like to create a remote GIS server. Let's discuss how roles and schemas work in PostgreSQL.

Roles are basically users that can log in, and do some stuff based on their permissions. Different roles can have different access levels to different databases. There are two kinds of roles--login roles and no-login roles (group roles). Group roles act as groups in operating systems; therefore, they can group multiple login roles, and manage their permissions in one place. Login roles are the typical users with passwords. The catch is, roles and group roles are independent from users and groups used by the operating system. System users cannot log in the database with their usernames, if a role was not created for them. Following this analogy, new users do not get roles created for them automatically by PostgreSQL.

Roles can have individual permissions, but they can also be managed by a group role. Similar to traditional users, roles can also have superuser (admin) capabilities. These superusers can modify roles and databases; therefore, they are quite dangerous to use as regular roles in a remote server. Additionally, they bypass every permission check, which makes them even more dangerous if exposed. The role postgres is a superuser role, which cannot be modified or dropped. It is completely fine to use in a local environment, as, by default, PostgreSQL does not accept connection requests from remote places, but only from the machine it is installed on.

It is still better to use a regular role in a local environment, as with a superuser role, you can accidentally overwrite sensitive values, drop tables, or, in the worst case, drop the entire database. You can also create a non-superuser role with privileges to modify databases and roles, and use the postgres role only if absolutely necessary.

Let's create two new login roles. One will be a regular GIS role with every kind of access to the GIS tables, while the other one will be a public role, which can only query the tables. It cannot modify them in any way; therefore, it will be safe to use by GeoServer, for example. We can create new roles by right-clicking on Login Roles, and selecting New Login Role. We must provide a name, and we should also add a password. As we do not want to create a superuser (having the postgres role is enough for that purpose), that's all we have to define. If we take a look at the last tab (SQL) after defining the required parameters, we can see the command that pgAdmin will use to create the role, which is as follows:

    CREATE ROLE gis LOGIN ENCRYPTED PASSWORD
'md53929f8e603334cb5a8c5a632bcc3f3ac'
VALID UNTIL 'infinity';

That is, it creates a role with CREATE ROLE, declares it as a login role with LOGIN, and, as it already calculated the hash of the password, it stores it directly with ENCRYPTED PASSWORD. It also includes a validity extent of infinity, which is superfluous.

If we wish to add a login role manually, we can simplify the command to the following:

    CREATE ROLE gis LOGIN PASSWORD 'mypassword';

In this case, the password is provided in plain text, and the hash is calculated by PostgreSQL. We can try this method out by creating the other role from the command line by opening an SQL window (Execute arbitrary SQL queries in pgAdmin 3 and Tools | Query Tool in pgAdmin 4). If the tool is disabled, first select a database by clicking on it. If we run the following query, we should be able to see the new role created:

    CREATE ROLE pubgis LOGIN PASSWORD 'pubpass';
Although MD5 is not considered a secure encryption method, by using a salt, PostgreSQL does a fair job in most of the cases. If you want access to better encryption algorithms, like bcrypt, you can use the pgcrypto extension; however, you also have to set up your own authentication system. You can learn more about pgcrypto at https://www.postgresql.org/docs/9.5/static/pgcrypto.html.

Now that we have some roles, we can give them privileges to administer or just query tables. However, doing so is very cumbersome, as these kinds of privileges do not apply on tables created later on. To solve this issue, PostgreSQL uses schemas to group tables and a lot of other things. In PostgreSQL, schemas are similar to folders in a file system. A database groups different schemas, while a single schema groups different tables. Schemas are like group roles for user management. The only difference is that using a schema is mandatory.

There are three different schemas--information_schema, pg_catalog, and public. The first two are system schemas used by the RDBMS internally, and showed as Catalogs in pgAdmin.

We should not alter those schemas in any way. The last is the default one in which we stored our data and PostGIS functionality. Having multiple schemas is a convenient way of organizing a big database, where only a single part contains spatial data. If we select the public schema, we can see the SQL command that creates it:

    CREATE SCHEMA public
AUTHORIZATION postgres;

GRANT ALL ON SCHEMA public TO postgres;
GRANT ALL ON SCHEMA public TO public;
COMMENT ON SCHEMA public
IS 'standard public schema';

Apart from the comment section, every command is important when creating a schema. We have to create it with CREATE SCHEMA, assign an owner with AUTHORIZATION, and give privileges to roles with the GRANT expressions. By using GRANT ALL, PostgreSQL automatically gives every privilege to postgres and public. The only problem is that we don't have a public role. By giving every permission to public, PostgreSQL implicitly says that every role ever created in this database should have every privilege to this schema. Sounds dangerous? It is very convenient though if used carefully.

You can revoke granted permissions with the REVOKE expression. If you would like to use the public schema in a safe way, you can use REVOKE ALL ON SCHEMA public FROM public; to revoke implicit privileges.

As a rule of thumb, we shouldn't keep any tables in the public schema which shouldn't be accessed and edited by every role. Let's put our spatial database in a dedicated schema with the following steps:

  1. Create a new schema by right-clicking on Schemas and selecting New Schema.
  2. Give a name to the new schema, and assign an owner. The owner should be the role we created for managing the spatial database (for me it is gis).
  3. Click on OK to create the schema.
  1. Open a new SQL window, and grant some schema privileges to the roles we created.
    • Grant every privilege to the administrator role by running the expression GRANT ALL ON SCHEMA spatial TO gis;.
    • Grant only select privileges to the public role by running the expression GRANT USAGE ON SCHEMA spatial TO pubgis;.
  1. In the SQL window, also grant some table privileges to the roles.
    • Grant every privilege to the administrator role with ALTER DEFAULT PRIVILEGES IN SCHEMA spatial GRANT ALL ON TABLES TO gis;.
    • Grant only select privileges to the public role with ALTER DEFAULT PRIVILEGES IN SCHEMA spatial GRANT SELECT ON TABLES TO pubgis;.
  2. Move every table to the new schema. Right-click on the tables in the public schema, and select Properties. There we can alter these properties. Watch out not to move the spatial_ref_sys table created by PostGIS. We can optionally set the owner of the tables to our GIS role:
Schema and table privileges differ; therefore, if you would like to fine-tune the privilege system, you cannot avoid using table permissions. On schemas, you can grant USAGECREATE, and ALL. For the rest of the privileges, you can read the PostgreSQL documentation at https://www.postgresql.org/docs/9.4/static/sql-grant.html. Make sure you select the appropriate documentation version.

We are all set. Let's try out our new schema by doing a spatial query that we've already done before:

    SELECT g.* FROM landuse l, geonames g 
WHERE ST_Intersects(l.geom, g.geom);

The preceding query returns an error. As we moved our tables out of the public schema, we have to explicitly define the schema of the tables with the syntax schema.table. If we update our query appropriately, PostGIS returns the intersecting features:

    SELECT g.* FROM spatial.landuse l, spatial.geonames g
WHERE ST_Intersects(l.geom, g.geom);

I'm completely sure you've already found out why PostGIS still works. As we left it in the public schema, its functionality remained exposed. Can we move it out to our new schema, and use its functions by prefixing them with the schema name? Not easily. PostGIS quite heavily relies on sitting in the public (or a similarly exposed) schema. However, as we witnessed, we can move out our spatial tables wherever we see them fit.

As we stated before, using the public schema is quite convenient. One of the reasons for this is that its content can be accessed directly without specifying the schema name. PostgreSQL achieves this by using a search path in which the public schema is defined. You can query the search path variable with the SHOW search_path; expression, and modify it with the SET search_path TO 'newsearchpath'; command. Adding additional schemas will make their content also available without prefixing. Be careful with this approach, though. Schemas contain local objects by design; thus, different schemas can share the same object (like table, function, and so on) names.

As the final task, let's open QGIS, and edit our PostGIS connection to use our new gis role. We have to use the database manager to get our layers again, as the project file for the previous chapter still thinks they are placed in the public schema, and, therefore, it cannot access them: