What are the differences between your image understanding and that of a computer’s?
Abstract:
High resolution Earth Observation images contain detailed information, making it possible to recognize objects within them. However, issues such as the sensory gap and the semantic gap cause difficulties for object recognition and labelling. The sensory gap has been defined as the difference between a real life scene and its sensory interpretation; while the semantic gap is considered as the difference between the user’s and the computer’s semantic understanding of objects in an image. This interactive tutorial will begin by introducing image mining systems, their goals, and the problems they suffer from. The focus will be on the sensory and semantic gaps and their causes, which will be highlighted with a visual demonstration. This will be followed by a quick experiment involving the audience. The results will be processed in real time, and analyzed jointly with the audience. Concluding this interactive tutorial will be a section concentrating on why these gaps are important to consider for practical applications, such as annotation tools, and image learning and mining systems.
Contents:
1. Introduction
a. Goal of image mining systems
b. Problems they suffer from:
i. Sensory Gap
ii. Semantic Gap
2. Causes of the Sensory Gap
a. Sensor type
b. Resolution
c. Perspective
d. Scale
e. Field of View (FoV)
3. Semantic gap
a. Types
i. User-User
ii. User-Computer
b. Causes
i. User background
ii. Additional information
iii. Difference between learning algorithms
iv. Difference between computer and human image understanding process
4. Quantification of the sensory and semantic gap (our approach)
a. Context annotation
b. Content annotation
c. User experiments
d. Comparison with computation models
5. Approaches to dealing with the semantic and sensory gaps in image understanding
6. Relationship between sensory and semantic gaps
7. Consequences for research in Earth Observation Image Information Mining
8. The importance of considering the sensory and the semantic gaps for practical applications