Skip to content

Commit

Permalink
update intro pages
Browse files Browse the repository at this point in the history
  • Loading branch information
rwood-97 committed Sep 27, 2024
1 parent 0e4d63b commit 0a64f9b
Show file tree
Hide file tree
Showing 5 changed files with 71 additions and 11 deletions.
Binary file added docs/source/_static/in_out_text_spotting.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
62 changes: 57 additions & 5 deletions docs/source/introduction-to-mapreader/what-is-mapreader.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,25 @@
What is MapReader?
===================

MapReader is an end-to-end computer vision (CV) pipeline for exploring and analyzing images at scale.
MapReader is an open-source python library for exploring and analyzing images at scale.

It contains two different pipelines:

- Classification pipeline: This pipeline enables users to fine-tune a classification model and predict the labels of patches created from a parent image.
- Text spotting pipeline: This pipeline enables users to detect and recognize text in map images.

MapReader was developed in the [Living with Machines](https://livingwithmachines.ac.uk/) project to analyze large collections of historical maps but is a _**generalizable**_ computer vision tool which can be applied to _**any images**_ in a wide variety of domains.

Origin of MapReader
-------------------

MapReader is a groundbreaking interdisciplinary tool that emerged from a specific set of geospatial historical research questions.
The classification pipeline was inspired by methods in biomedical imaging and geographic information science, which were adapted for use by historians, for example in our [Journal of Victorian Culture](https://doi.org/10.1093/jvcult/vcab009) and [Geospatial Humanities 2022 SIGSPATIAL workshop](https://arxiv.org/abs/2111.15592) papers.
The success of the tool subsequently generated interest from plant phenotype researchers working with large image datasets, and so MapReader is an example of cross-pollination between the humanities and the sciences made possible by reproducible data science.

Since then, MapReader has expanded to include a text spotting pipeline, which enables users to detect and recognize text in map images.

.. TODO:: Add info here about the text spotting pipeline

What is unique about MapReader?
--------------------------------
Expand All @@ -15,18 +33,32 @@ This unique way of pre-processing map images enables the use of image classifica
What is 'the MapReader pipeline'?
---------------------------------

The MapReader pipeline consists of a linear sequence of tasks:
MapReader now contains two different pipelines:

- Classification pipeline: This pipeline enables users to fine-tune a classification model and predict the labels of patches created from a parent image.
- Text spotting pipeline: This pipeline enables users to detect and recognize text in map images.

Classification pipeline
~~~~~~~~~~~~~~~~~~~~~~~

The classification pipeline was the original 'MapReader pipeline'.
It enables users to train a classification model to recognize visual features within map images and to identify patches containing these features across entire map collections:

.. image:: /_static/pipeline_explained.png

Together, these tasks can be used to train a computer vision (CV) classifier to recognize visual features within maps and identify patches containing these features across entire map collections.
Text spotting pipeline
~~~~~~~~~~~~~~~~~~~~~~

The MapReader text spotting pipeline enables users to detect and recognize text in map images using a pre-trained text spotting model:

.. image:: /_static/text-spotting-pipeline.png

What kind of visual features can MapReader help me identify?
------------------------------------------------------------

In order to train a CV classifier to recognize visual features within your maps, your features must have a homogeneous visual signal across your map collection (i.e. always be represented in the same way).
In order to train a CV classification model to recognize visual features within your maps, your features must have a homogeneous visual signal across your map collection (i.e. always be represented in the same way).

What are the inputs and outputs of each stage in the MapReader pipeline?
What are the inputs and outputs of each stage in the MapReader classification pipeline?
------------------------------------------------------------------------

Download
Expand All @@ -48,3 +80,23 @@ Classify (Train and Predict)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. image:: /_static/in_out_classify.png
:width: 600px

What are the inputs and outputs of the MapReader text spotting pipeline?
------------------------------------------------------------------------


Download
~~~~~~~~
.. image:: /_static/in_out_download.png
:width: 600px

Load
~~~~
.. image:: /_static/in_out_load.png
:width: 600px

Spot Text
~~~~~~~~~

.. image:: /_static/in_out_text_spotting.png
:width: 600px
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,4 @@ What skills/knowledge do I need to use MapReader?
- Basic Python
- Basic understanding of machine learning and computer vision (CV) methodology

..
Add something here about how we have resources for you to learn these skills.
Users who are interested in using MapReader but do not yet have these skills are encourage to refer to our :doc:`in-depth-resources/coding-basics` section for resources on learning these skills.
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ Who might be interested in using MapReader?

MapReader might be useful to you if:

- You have access to a large collection of maps and want to identify visual features within them without having to manually annotating each map.
- You have access to a large collection of maps and want to identify visual features or text within them without having to manually annotate each map.
- You want to quickly test different labels to help refine a research question that depends on identifying visual features within maps before/without committing to manual vector data creation.
- Your maps were created before surveying accuracy reached modern standards, and therefore you do not want to create overly precise geolocated data based on the content of those maps.
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,16 @@ This exact number will vary depending on:
- the skills you (or your team) have,
- the amount of time at your disposal.

Deciding to use MapReader, which uses deep learning computer vision (CV) models to predict the class of content on patches across many sheets, means weighing the pros and cons of working with the data output that is inferred by the model.
Inferred data can be evaluated against expert-annotated data to understand its general quality (are all instances of a feature of interest identified by the model? does the model apply the correct label to that feature?), but in the full dataset there *will necessarily be* some percentage of error.
MapReader uses computer vision (CV) models to extract information (class labels and text) from map images.
This enables users to generate datasets for large corpora of maps in a fraction of the time it would take to annotate them manually.

MapReader creates output that you can link and analyze in relation to other geospatial datasets (e.g. census, gazetteers, toponyms in text corpora).
If georeferencing information is available for the map images, MapReader can create georeferenced outputs that can be linked and analyzed in relation to other geospatial datasets (e.g. census, gazetteers, toponyms in text corpora).
This allows users a new way to explore and analyze their map collections.

Understanding the limitations of MapReader
------------------------------------------

Deciding to use MapReader means weighing the pros and cons of working with data that has been inferred by a computer vision model.

This inferred data can be evaluated against expert-annotated data (i.e. ground truth data) to understand its general quality, but users should be aware that in the full dataset there *will necessarily be* some percentage of error.
As such, MapReader may not be suitable for users who require completely accurate data.

0 comments on commit 0a64f9b

Please sign in to comment.