Semantic understanding of high spatial resolution remote sensing (RS) images can be divided into object detection, object labelling, identification of geospatial relationships, and semantic description generation. Geographical relations represent the spatial distribution dependencies between geospatial entities such as points, lines, and polygons, and the topologies among them. Geospatial relations play a very important role in describing the relations between geographic objects. These relations can be broadly classified as topological, directional, and proximity relations. These relations describe the adjacency and association relations between geospatial objects. An approach to identify an appropriate directional geospatial relationship between geo-objects present in high spatial resolution RS images is proposed in this paper. Geospatial objects in the form of the closed boundary are taken as input and relationship triplets are generated. Two approaches have been used in the identification of directional relationships and the results of both approaches are compared. The first approach is based on the centroid of the objects and the second considers whole objects while calculating the direction. These relations are then further represented using a knowledge graph, where nodes represent objects and edges represent their relationship. Knowledge graph plays a very important role in overall scene understanding. It shows the association of all objects with each other. These relationships are then represented in the form of descriptions by using template-based sentence generation. Results show that these directional relationships are accurately identified between each pair of objects using both approaches, but relations generated by considering whole objects are closer to human cognition. Semantic under-standing of remote sensing images is of great significance in different applications such as urban surveys, urban planning, and management, military intelligence, etc.