Gerhard Jaritz is Senior Research Fellow at the Institut füaut;r Realienkunde at the Austrian Academy of Sciences, Krems, and Professor of Medieval Studies at Central European University, Budapest.

Bound Images: Encoding and Analysis

Gerhard Jaritz (Krems / Budapest)

For any analysis in comparative historical - and not only historical - research, context and contextuality are of major importance. This is particularly true for any analysis of image information. The example I would like to concentrate on is part of the work and research done at the "Institut fuer Realienkunde des Mittelalters und der fruehen Neuzeit" (Institute for the research into daily life and material culture of the Middle Ages and the early modern period) of the Austrian Academy of Sciences that uses the DBMS "kleio IAS (Image Analysis System)" having been developed at the "Max-Planck-Institut fuer Geschichte" in Goettingen (Germany).

The mentioned image analysis project has been based on very detailed structured verbal descriptions of images using controlled and standardized terminology, and/or transcriptions of texts (of material like charters, manuscripts etc.). Concerning the analysis of transcribed text markup is indispensable for converting it into components of a structured data base into which each relevant part of the transcription is to be parsed after the markup has been completed. Those various texts are connected with the appropriate digitized images. The verbal descriptions and transcriptions have been done while being conscious that "language is not very well equipped to offer a notation of a particular picture. It is a generalizing tool" (Baxandall, 3). Many aspects of the "pictorial grammar, based upon current rules for the representation of images within a given culture" (Stenvert, 28) still remain rather indescribable through words.

Textuality, contextuality and context open up a wider range of opportunities for source-oriented, qualitative and quantitative image research that may use different links of source information in various ways. Often, the interpretation of visual data requires the possibility to refer to parts or parts of parts of an image or a manuscript page corresponding to a part of the image description or the manuscript transcription. Links become necessary. Such links are mainly reached by using features of digitized image binding. Each portion of the verbal text (portion of controlled or standardized verbal text describing the images, or part of the transcription of digitized text) is explicitly related to a sector within the appropriate image, within the bitmapped object. By delimiting and binding, and that way encoding the relevant segments of the image and linking them to the referring portions of the verbal text, one receives sectors and connections which will be available for any further comparison and analysis. In the course of binding, coordinates are created which become part of the structured text data base in elements directly connected to the parts of text which are corresponding to the image segments. They are of high relevance concerning data exchange. The bindings or the coordinates of them respectively can also be kept in evidence even if the data base gets damaged and/or has to be created again.

Fig.: Raw data with automatically created coordinate encodings of bound image segments:

I $ archivnr = 000301 / slidenr = 7000001-7000003 / negativenr = 5000047 ; 3000600-3000603 ; 3000663 / digtotnr = 000301 / digdetnr = 000301-1 ; 000301-2

W $ form = panel painting / type = panel of the Schotten-altar / artist = master of the Schotten-altar / title = Crucifixion of Christ / provenance = Vienna

D $ post = 1469.000000 / ante = 1480.000000 / place = Vienna / county = Vienna / institution = Schotten Abbey

N $ origin = Vienna, Schotten Abbey

S $ action = crucifixion / setting = landscape ; Mount Calvary

P $ sex = f / gesture = mourning / digtotnr = 000301:5,159:94,594

C $ name = cap / color = white

C $ name = heuk / color = green / digtotnr = 000301:0,261:123,606

C $ name = gown / color = blue

P $ sex = f / gesture = mourning / digtotnr = 000301:67,121:188,392

C $ name = veil / color = white / digtotnr = 000301:83,145:162,308

Out of that connection of text and image details, possibilities for image analysis are available that are based on delimited verbal texts (= e.g., part of the image description) and their connection to the appropriate segmental image information. These are links which allow to move from any of the delimited categories (groups, fields, attributes) of the verbal text in the database to the graphical representation of those parts of the source from which the categories were derived. It is certainly also possible to concentrate on a portion of the digitized object and go from there to its "interpretation" (= the verbal text). Overlappings are possible and can be dealt with in a kind of dialogue policy. Generally, the exploratory treatment of bitmapped source and verbal texts obtain increasing relevance.

Having bound, e.g., the describing text "drinking glass" on (= in relation to) "table" to its graphical representation, i.e. the appropriate sector of a digitized image on which the topic is depicted, and asking now for all depicted "drinking glass(es)" on "table(s)", offers that image sector and its connection to the segmental verbal text. We have the opportunity to connect it with background knowledge about drinking glasses or about tables. We can try to apply substraction- and similarity-routines for image sector comparison. At the same time the connection of those image sectors and the delimited texts with the appropriate complete reference images and the complete text of their descriptions stays active. The context of image, of verbal text being associated to this image, of image segment and of text associated with that image segment are available and ready for comparative analysis. Such bindings are often also very advisable in connection with the application of various features of image enhancements that help to recognize and to interpret fuzzy portions of images or digitized text.

Moreover, the creation of so-called segment-archives (e.g., the "drinking glass(es)" on "table(s)" segment-archive) opens the possibility for very detailed comparisons of related digitized image-segments, while keeping all the before mentioned links to reference images and texts available. Those digital typological segment-archives play a particular role in the course of any comparative image analysis. Possibilities to create and administrate such "archives" to collect and to compare larger numbers of various individual segment information on different levels in the course of analysis are highly relevant in the course of the research process. We can decide if we want to have 9, 25, 49 or 81 of such digital image sectors out of our database for comparison on one screen, the latter number particularly used for paleographical research and the comparison of letters in digitized texts.

Out of segment-archives the creation of segment-subarchives is possible. The example of a "red dress"-segment-archive, e.g., might be mentioned which contains the segments of all persons in a database of late medieval images who wear red dress. Out of the reason that "red dress" might have born positive as well as negative connotations of the person, the creation of adequately context-bound segment-subarchives became necessary, a "positive red dress"-segment-subarchive and a "negative red dress"-segment-subarchive.

That means generally that bound images, their various manipulations, and any kind of analysis of them are to be found or are taking place respectively in an environment of relevant contexts. Therefore, such "encoding" of digitized images by applying semantically bound digitized image tagging is a valuable basis and means for any kind of comparative visual - and not only visual - image content analysis.

References

Michael Baxandall, Patterns of Intention. On the Historical Explanation of Pictures, (New Haven, London, 1986)

Jurij Fikfak, Gerhard Jaritz (eds.), Image Processing in History: Towards Open Systems, (St.Katharinen, 1993), (= Halbgraue Reihe zur historischen Fachinformatik, A16)

Daniel J. Greenstein, "Encoding Standards for Computer-Assisted Historical Research: the Problems Reasssessed," in: Daniel J. Greenstein (ed.), Modelling Historical Data: Towards a Standard for Encoding and Exchanging Machine Readable Texts, (St.Katharinen, 1991), 1-18, (= Halbgraue Reihe zur historischen Fachinformatik, A11)

Gerhard Jaritz, Barbara Schuh, "Describing the Indescribable," in: Manfred Thaller (ed.), Images and Manuscripts in Historical Computing, 143-153

Gerhard Jaritz, Images. A Primer of Computer-Supported Analysis with kleio IAS, (St.Katharinen, 1993), (= Halbgraue Reihe zur historischen Fachinformatik, A22)

Gerhard Jaritz, "Comparative Analysis of Historical Image Information," in: Axel Bolvig (ed.), Electronic Filing, Registration, and Communication of Visual Historical Data. Abstracts for Round Table 34 of the 18th International Congress of Historical Sciences, (Goettingen, Montreal, 1995), 29-34

Ronald Stenvert, Constructing the Past: Computer-Assisted Architectural-Historical Research, (Utrecht, 1991)

Manfred Thaller (ed.), Images and Manuscripts in Historical Computing, (St.Katharinen, 1992), (= Halbgraue Reihe zur historischen Fachinformatik, A14)

Manfred Thaller, "The Need for Standards: Data Modelling and Exchange." In: Daniel J. Greenstein (ed.), Modelling Historical Data, 1-18

Manfred Thaller, kleio. A Database System (St.Katharinen, 1993), (= Halbgraue Reihe zur historischen Fachinformatik, B11)