AN INTRODUCTION TO CROSS MEDIA MAPPING

     (Based on a reading given at the '6e Journee Internationale d'etude de
         Musique Electroacoustique', June 1976, in Bourges - France)
 
 
 

 

      This remains a fairly tautological definition, but a moments
      consideration will show that any description or definition, must either
      rely on undefined terms and thus remain incomplete, or else become a
      closed definition - and therefore tautological.

      It is a strong personal belief that tautology plays a fundamental role
      in communication systems, but the precise nature of this role, and the
      question whether a tautological statement can be of value or not - will,
      like so many other questions, temporarily at least, remain open.
 

      Such a definition for language as just stated, includes natural
      languages, artificial languages, and possibly animal communication
      systems (dependant on how strongly the conditions requiring the
      production of compound signs is interpreted) but it certainly includes
      art languages in the form of painting, sculpture music, drama, etc.

      These "art languages" are often divided from "verbal languages on the
      grounds that the latter have specified interpretations for the
      repertoire of signs while the former have no specific interpretations.

      This division is perhaps worthy of further investigation.
 

      It was stated earlier that although the test that communication had
      taken place involved a satisfactory or correct response. This should not
      be taken as implying that a simple stimulus -> response mechanism was
      operative.

      Even with verbal languages, where the interpretation of individual signs
      is assumed specified, often several exchanges need to be made before
      communication can truly be said to have taken place.  Sometimes new
      words are introduced, or existing words need clarification.  Statements
      and answers are gradually modified until the communicants are reasonably
      satisfied that they have approximately the same idea of what was said.

      However, a satisfactory answer is not always elicited, and sometimes
      even highly unsatisfactory responses may result.

      In some cases this results from a difference of opinion (the definition
      of which will also be left open), but often it is the result of a
      genuine disagreement over the interpretation of a specific sign or group
      of signs.

      Sometimes, the difference between opinion and interpretation may be
      difficult to distinguish, for example with such concepts as art,
      democracy, freedom, etc., opinions regarding their modes of operation
      and usefulness are almost inextricably interwound with the definition of
      the word.

      On the other hand, statements regarding whether a walk in the rain is
      pleasant or not, are purely a matter of opinion, and independent of
      interpretation problems in the sense just mentioned.  In such cases as
      these, judgement of a satisfactory response must be more in terms of it
      appropriateness to the context than to a correct opinion.

      Interesting and complicated as the precise relationship between opinion
      and interpretation may be, there is insufficient time to discuss it
      further.  However the fact remains that within a so-called linguistic
      community there are differences in interpretation.

      For example, in American English "sidewalk" is equivalent to "pavement"
      in English english; while "pavement" in American English is equivalent
      to roadway in English english.

      This has the result that an American and an Englishman will produce
      entirely different responses to a sign composed of the words
      "automobiles must be parked on the pavement"

      These differences in meaning are due either to different interpretations
      being assigned to the same physical signal, or to apparently equivalent
      interpretations being assigned to different signals, imply that the
      general linguistic community of "English speaking people" needs to be
      further divided into subcommunities which may be referred to as "English
      dialects".
 

      But does a dialect deliniate the smallest group within which there are
      no more variations between signal and interpretation?

      Perhaps a personal experience here can answer the question.  One day
      while talking to my mother I was surprised to hear her suddenly say
      "Hey, look at that silly tit hanging upside-down nibbling his nuts."

      Being a little shocked by this remark, I looked in the direction she
      was pointing and seeing a bird feeding in the garden, realised that she
      and I had interpreted the sentence in two completely different ways.

      It would seem that the sub-category of dialect may need to be further
      subdivided into "idiolects".  In other words, the linguistic community
      may have only a single member.

      Should this appear to be a hasty decision to be made on the basis of a
      simple example perhaps it can be justified by considering variations in
      "style" and the use of personal idioms which most people use in their
      speech.
 

      It may even be possible that the variety of psycho-linguistic theories
      is not a result of misunderstanding a single objective reality, but the
      result of differences in the linguistic strategies used by the authors.
      In other words, because their individual use of language is different,
      their theories are different.

      Or, one could simply ask why do so many misunderstandings occur, and why
      is it sometimes so difficult to communicate verbally, if both
      communicants are using identical rules to assign and combine
      interpretations.

      Another basic assumption that is used to divide art-languages from
      verbal languages is a belief in the existence of objective concepts
      external to the language, the purpose of a specific verbal language
      being to communicate these concepts.

      It may be debatable whether or not a musical language refers only to
      musical concepts, but the belief that it is possible to translate, for
      example from French to English, or vice-versa appears to imply the
      existence of a something that can be translated.  A significance, as it
      were, that only needed to be assigned to a new signal in order to be
      translated.
 

      Early attempts at machine translation of texts soon showed that
      translation was not a matter of interchanging words with similar meaning
      and arranging them in the correct grammatical order.

      For example, for my own interest, I recently used a dictionary to
      translate all the meanings of the Dutch word "opnemen" into English,
      then each English equivalent back to Dutch, and finally back to English
      again.

      The most general translation for "opnemen" is; to take up, which can
      also be found in the construction of the word;
      i.e. op => up, nemen => take.

      However, also listed is the translation "take down (stenography)" as in
      the English sentence, I'll just take down your address!

      The apparent contradiction between "take up" and "take down" can be
      resolved by realising that simultaneously the address is both "taken up"
      in memory, and "taken down" on paper.  In this case the piece of paper
      is also the memory.

      Clearly, the focus of attention is on one of two different aspects of
      the same action, or in other words the action is seen from two different
      viewpoints.

      Another translation for "opnemen" is "collect (votes)".  The translation
      for collect is "ophalen", which in turn generates; draw up (a bridge),
      pull up (blinds), raise (a curtain), weigh (anchor), shrug (ones
      shoulders), turn up (ones nose), collect (money).

      At first sight it would appear that English speakers like to make life
      more difficult by using different words to specify the same activity
      when performed with different objects.

      A closer inspection shows that while in some sense this is true, it does
      not apply to all cases - for example, to 'draw up', to pull up,' to
      'raise', or even to 'weigh' a bridge are to some extent interchangeable,
      and can convey the idea of a bridge being lifted, even if some
      flexibility of interpretation is required for individual words.

      On the other hand, the phrases, to 'shrug a bridge', to 'turn up a
      bridge' or to 'collect a bridge', apart from having a surrealistic
      effect, produce images that are both radically different from each
      other, and from the previous set of phrases

      If there were external concepts, existing outside the language -a kind
      of disembodied meaning -waiting to be assigned to a signal in order to
      be communicated. Then it could be expected that a simple relationship
      would be apparent between words of different languages.

      Instead, one finds complex networks of meaning that are impossible to
      relate exactly to each other.  In fact, there are intersections in one
      place, and not in another, the network may be shifted to produce a
      better relation in one area, but the difference grows wider somewhere
      else.

      Bearing in mind that Dutch and English are closely related languages,
      this basic mismatch between the two conceptual networks would appear to
      be a serious threat to the belief in a set of external translatable
      concepts.
 

 

      If cognitive processes are involved in perception, it is not
      unreasonable to assume that perceptual information plays an important
      role in cognition; in fact if divine intervention is ruled out, then the
      knowledge of the environment essential for survival, can only be
      obtained through the sense by communication with others, or as a result
      of some kind of deductive thinking.  Should cognition and perception
      prove to be inter-related, then a tautological system would seem difficult
      to avoid.

      However, this is a somewhat advanced point in my argument.  At present I
      will be satisfied with having drawn attention to the fact that verbal
      language not so safe and efficient as our daily reliance on it would
      appear to presuppose, that it is in fact concerned not with the simple
      assignment of external concepts to a pre-agreed signal set, but with the
      more fundamental task of searching for intersections between separate
      and individual logic systems, that are in turn based on varying
      differences in perceiving the world in which the authors of these
      systems live.

      But what relation does this have with art, or, to be precise, music?
      Well one hardly needs to be reminded that the concept music also covers
      a wide range of different sub-categories that can also eventually be
      reduced to a group with a single member - i.e. the individual composer
      or musician
 

      Each composer has his own definition of music, his own concepts of how
      it operates, how it should be made, what the major problems are, and how
      they should be solved.  This is to be expected, we live in a time of
      exploration and expansion of new concepts, but we shouldn't let this
      delude us into thinking that music is by definition a set of systems
      with non-explicit interpretations and therefore automatically opposed to
      the explicit codings found in verbal communication.

      I have tried to draw attention to the fact that verbal systems are not
      so explicit as at first might be imagined, but conversely, it should not
      be forgotten that although it is now unfashionable for music systems to
      have a set of specific semantic assignments it is no more or no less
      impossible than with verbal languages.  Perhaps the two communication
      systems are not so radically different.
 

      Until now, focus has been on the difficulties of communication, and the
      impression may have been created that I am trying to prove that it
      impossible.  Clearly communication does in fact occur.  Or at least the
      condition that a satisfactory response should result from the
      transmission of signals is sometimes fulfilled, and so it can be assumed
      that communication has taken place.

      Before the objection is made that this response test is not applied in
      music, and therefore by my own criteria music is not a communications
      medium.  I refer to musical feedback as manifest by discussion,
      compositions by other composers, or improvisations.
 

 
 

      If concepts are influenced by the description language, it cannot be
      neutral.

      On the negative side, this implies that distortion and misunderstanding
      are not only to be expected, but almost unavoidable when communicating.

      On the positive side, these "distortions" may prove beneficial in
      breaking through tautologies and generating new insights into old
      problems.

      This is of course a rather large and vaguely defined area in which to
      explore. The rest of the paper will therefore be concerned primarily
      with the construction of descriptions, with the intention of building a
      foundation for further investigation of the specific properties of
      descriptions, and their interaction.
 

      There are perhaps three basic strategies for constructing a description,
      which produce the following basic types:

  1. An Operational Description:

         i.e. a description in terms of the operations performed on;
         or with the subject, or operations performed by the subject.

         These descriptions are often in terms of internal motor-sensory
         mechanisms, but may be verbalised by such statements as "it's used for
         opening tins".

         However the same strategy is also in formal logic systems where a set
         of axioms may be used to describe an operator in terms of the change its
         effect on the elements.  In other words a truth table in two valid
         logic is in fact a description of the operator.

         While being aware of the apparent confusion between description and
         definition, I feel that this is a problem that can be solved best
         at a later date.
 

   2. An Analytic description:

         i.e. a description in terms of basic features or qualities that
         the subject is assumed to have.  Such a description would be
         "a large hairy ball with green ears."

         For completeness, relationships like above/below or greater than/smaller
         than are also included as features.

         It may also be useful to distinguish between perceptual analysis where
         the "sensory image" of the subject is described, and a conceptual
         analysis where the sensory analyses have been co-ordinated and projected
         onto the subject.

         The psychological difference can perhaps be seen in visual art by
         comparing a painting with apatial perspective - which is perceptual,
         an icon, where visual proportions are derived from conceptualised
         relations such as "the most important or least important figure
         in the picture".

         Both the history of western painting and the development of visual
         skills in young children suggest that conceptual analysis is developed
         before explicit perceptual analysis.
 

   3. An Analogical Description:

         i.e. something is described in terms of something else.
         Such a description would be "just like an apple".

         The difference between metaphor and analogy is so slight that I propose
         to ignore it, but it is not unreasonable to suggest that all conceptual
         or physical models are considered to be examples of analogy.

         To the above comments, can be added the remark that given a specific
         signal there is no a-priori reason for a particular strategy to be
         used.  The signal may be analysed, it may be given an analogical
         description, for example, "it's just like a scream", or it may be
         described by its effect, for example, "it gives a pain in my ears".
         All three descriptions may even be combined to form a single concept
         of the event.

 

      It can also be seen that a "nesting of descriptions" occurs, i.e. to say
      "it's just like an apple" implies that a description of an apple exists.
      Or an analytical description may be in terms of components as for example
      in a wall which is made of bricks, where the bricks would represent
      another level of description.

      This process is also echoed in formal systems, where for convenience
      definitions are used as abbreviations by substituting a long statement
      by a shorter statement or label.

      The "nesting" of descriptions reintroduces the problem of tautology by
      reason of the fact that our conceptual image of the world would appear
      to be either based on an infinite regress of definitions, or else these
      definitions must in some sense be mutually supportive, and therefore
      tautological.
 

      Seen from another standpoint, these interrelations between descriptions
      may be considered as being similar to computer sub-programs, not only
      because they are necessary means of organising vast quantities of data
      and frequently recurring operations, but also by virtue of the fact that
      one small error within one of these sub-structures could have drastic
      consequences regarding the performance of the whole structure.
 

      However, the area of investigation at present is primarily concerned
      with the construction of descriptions, and here too a problem of
      infinite regression appears.

      The problem is that an analytical description would seem to be a
      prerequisite for the other description strategies.
      For example, unless there is an analytical description for two objects,
      or situations, which shows them to be alike - how can one form an
      analogy for the other?

      But if an analytical description is required, it becomes difficult not
      to ask how the concepts that form the basis of the analysis are
      distinguished from each other without the use of another analytical
      description language.
 

      On the other hand, if analogy is taken as being a "basic description
      principle" it could be said that if A can be used as an analogy for B,
      then A is a description language for both A and B, so the condition that
      a common description for both is necessary, becomes fulfilled -without
      the need for analysis.

      Should this seem like an irrelevant linguistic trick, a cursory glance
      at child development would show that a child can correctly assign the
      label "tea cup" to an object, long before he can say "hollow thing, with
      a handle, used for drinking tea!" So it would seem that identification
      can be operative without explicit description.

      It can also be seen that any individual teacup can be used as an analogy
      for any other.
 

 

      The problem, however, is not completely solved.  Still remaining is the
      question of how it is possible to separate the constant flow of sensory
      information into discrete objects without the use of analysis.  The
      assumption being that without two discrete objects it would be
      impossible for one to form a description of the other.
 

      Before going further, it is necessary to make a distinction between the
      "implicit" language actually used by an organism, and an explicit
      description such as would be necessary for an observer of the organism.

      The situation is similar to that of a machine with an observer, the
      machine reacts to its inputs and exhibits certain states- an observer
      would require a description language to refer to the activities of the
      machine, while the machine is able to function without a description of
      itself.
 

      It the system has a quantitative input, and produces a non-quantitative
      output (i.e. the system responds in different ways - instead of an
      increase or decrease of a single reaction) -then it could be argued that
      an explicit analysis of the input is required, and that this presupposes
      a description language.

      While this may be true for a sequential machine, the parallel use of
      detectors similar to band-pass filters should permit the automatic
      analysis of as many states as it was desirable to decode.

      A similar mechanism is in fact found in the eye, where special cells, by
      only reacting to certain frequencies of electromagnetic waves are able
      to generate the impression of colour.

      Note, the emphasis is on generation of colour, and not detection of
      colour. The reason for the generation is easy to understand if one
      compares the task of trying to remember a specific tone of grey, with
      the task of trying to remember a specific colour.

      While not having done the necessary research, I suspect that the other
      sense organs work on similar principles.

      But the question of how to isolate discrete objects remains.  This could
      perhaps be solved by use of a decaying memory trace which needed to be
      refreshed constantly -as a filter for "frequently recurring" and "not
      frequently recurring" sensory patterns.

      If a filter like this was used in conjunction with the system of
      band-pass type filters just described, the relatively constant sensory
      pattern remaining in the memory would form the object, while noise and
      transitory effects would be automatically lost.
 

      Association and conditioning would be fairly simple to explain within
      such a model, due to the fact that the entire contents of the memory
      would represent the object, and only later experience would show which
      implicit features were relevant, and which were irrelevant.

      It may be possible that concepts such as cause and effect, and
      implication are at least partly derived from this basic mechanism.
 

      Assuming that such a system was able to correlate between the various
      sets of sensory information by reducing all information to a common
      internal code, then it should be possible for the organism to
      distinguish between external "tangible" space and its own internal
      "intangible" space; "objects" and "operation" would then be definable
      in terms of non-motor sensory co-ordinates and motor-sensory co-ordinates,
      and therefore only expressible in terms relative to the organism itself
      (as observed by Piaget in young children).

      This is only a rough sketch, and is not intended to deny the role of
      motor-sensory co-ordinates in perception, but it does indicate that at
      least in theory the apparent preconditions for analogy can be satisfied.
 

      Although it is impossible to state when, or why, speech developed - the
      association of verbal labels with sensory co-ordinates should be a less
      difficult problem than that which it provides for later generation,
      namely the problem of decoding the exact interpretation of a verbal
      label.
 

      The fact that any member of a class of objects, may be used as an
      analogy for any other member of that class, when combined with the
      constancy of a verbal label as opposes to the transitory nature of the
      sensory co-ordinate to which it refers - could be a source of complex
      philosophical problems, such as the existence of paradigm cases, etc.

      The Classical Greek preference for the conceptual label and their
      dislike of the troublesome sensory information, which formed the basis
      of the split between Western and Eastern modes of thinking may have lead
      to scientific advances -but we must remain wary of the danger of
      becoming trapped in our own descriptions.
 

      Any conceptual systems should be capable of dividing the observed
      phenomena in the world into two categories, i.e. phenomena that are
      explainable and those that are not.

      Logically it is impossible to explain the unknown, except in terms of
      the known.
 
 

      Primitive analogy has one great limitation as a method of explanation
      namely it is impossible to specify in which sense the analogy is true.
      For example, a statement such as "a cabbage is like a rose" could cause
      considerable misunderstanding which could be removed by stating "a
      cabbage has a similar structure to a rose."
 

      Suddenly the old problem reappears, how is the concept 'structure' to be
      derived without an analytical description language, and how are the
      concepts that form this language derived without an initial starting
      point-

      Without going into complex detail, I would suggest that these concepts
      are derived initially from a description of man's own non-directed play
      activities, after the discovery that he had by accident "made
      something".

      In other words, the process of making a model comes first, and only
      afterwards is it discovered what this is a model of.
 

      For example, a man idly making marks on a surface with a burnt stick,
      suddenly discovers it "looks like" a bull; by further experiment it
      becomes possible to draw a lion etc.  His own activities would then
      present the controlled situation necessary to develop a description
      language, and he would be able to analyse the real subject in terms of
      the operations involved in producing the model.
 

      Initially, because his basic conceptual system was operating on
      principles of analogy the possibility of confusion between the actual
      object and the image would be great.

      This would lead to the development of "sympathetic magic", the power of
      which would provide the incentive to continue experimenting with the
      production of models, which would in turn generate more analytical
      concepts.
 

      However, because I have given a visual example, it must not be implied
      that these are purely visual processes.  It would seem impossible to
      analyse pitch structures, for example, until pitch had been identified
      as being a function of the dimension of the sound-producing object, even
      to the point that the relative lengths of these objects form the basis
      on which the pitch structures are organised.

      What is implied, is that the invention of written music would be almost
      impossible without the prior invention of instruments-irrespective of
      whether the notation was specific to the instrument, or mapped into a
      single hypothetical "objective" space as occurred in the West.

      Also implied, is that knowledge of the method of sound production is
      important for the "syntactic" interpretation of sound structures, for
      example on the simple level of distinguishing the interplay of elements
      by correctly assigning timbral qualities to their sources.  One can
      therefore suspect that electronic tape music will continue to be an
      exotic science-fiction sound effect for the majority until the public
      has had more experience with the means of production.

      It is also possible that in instrumental music, knowledge of the
      motor-sensory activities involved in sound production may be linked in
      some cases to "semantic" interpretation.
 

      The availability of analytical description permits the development of
      experiment - which basically consists of manipulating a described object
      and testing to see of there is a change in the description as a result
      of the operation.

      This constancy, or lack of constancy, in the description before and
      after the operation, is of fundamental importance, and forms a basis for
      defining the geometry of the system under investigation.
 

      In fact, in formal systems, a geometry is defined by the changes in the
      description space that result from the application of an operator.
      These changes being defined by the construction of the axioms.

      Experiment is, therefore, essentially a search for the geometry of the
      system.
 

      The role of the description language is of vital importance under these
      conditions, for example, an unchanged description may mean that the
      subject is unchanged, or, that the description fails to register the
      change.  It becomes impossible to prove that the object has not changed,
      and only possible to search for a new description, or to admit that a
      specific time no such language exists.

      It can be seen in this context that the most difficult task for the
      public when faced with contemporary music is to decide which
      description, out of a wide range of possibilities, should be used to
      interpret the specific composition they are listening to.

      Also implied is that a search for a single meta-language with which to
      analyse every type of music is in fact a misdirected task, and that a
      phenomenological approach related to the actual description used by the
      composer would be more useful.
 

      It may happen, with an experiment, that a feature that is detectable but
      not describable is generated by an operation.  Such a feature can only
      be given an operational description in terms of the specific situation
      in which it was first discovered.

      The next step is to test other descriptions to see if they can be
      applied to the problem of isolating the new feature.

      Assuming that such a language is found, this feature forms an
      intersection between the old operational description and the new
      analytical description so that a new item of knowledge is generated
      -namely operation X results in a change in parameter Y.

      Thus knowledge in this case, resulted from the discovery of
      "incompleteness" in the original description (or the "inconsistency"
      inherent in a situation that is both changed and unchanged) - the
      application of a new description, and the final integration of the
      different descriptions in order to resolve the original inconsistency.
 

      In this sense, "objectivity" clearly refers not to a "real world" but to
      the ability to make correlations between different 'description languages'.
      So that the distinction between 'scientific' and 'non-scientific' models
      may be found in the degree of consistency within the model and the
      possibility of correlation with other descriptions.

      Under these conditions, if all known explanations can be mapped into a
      single description or a single description can explain all known
      phenomena, then this description is assumed true.

      It can clearly be seen, that if all known phenomena can be explained in
      terms of 'God', the 'ether', the 'orbits of electrons', or little green
      men from Mars, then these concepts are assumed true, until proved
      incomplete or inconsistent.
 

      Until now, consideration has been given to the development of conceptual
      geometries based on physical experiment, or derived form physical signal
      patterns such as might be manifested by a musical structure or a visual
      composition.
 

      However, the verbal labels used for objects and operations, by reason of
      their abstract nature, need to be manipulated by rule, as opposed to the
      physical operations of earlier systems.

      The rules by which abstract systems operate must presumably be derived,
      initially, from observations and experiments in the physical world.

      However, another characteristic of such formal systems, is, that instead
      of preceding from observation to hypothesis, the procedure can be
      reversed:

      Thus the basic mode of operation becomes "if a system (or geometry) was
      based on these basic principles (or axioms) then this would be the
      result."
 

      As demonstrated earlier, the removal of inconsistencies and incomplete
      descriptions not only leads directly to an increase in knowledge but
      also to tautology.

      For example, if a formal system is used to describe all musical
      structures, and this description is then used by composers, the prophecy
      becomes self fulfilling -as for example with traditional tonality
      -which was presumably assumed to be an objective system.
 

      Although, to a certain, ambiguity may be useful in breaking out of
      tautological systems, this is not encouraged in formal systems -but
      they do have one advantage.

      Because they are not bound by physical laws -it is possible to change
      the rules (or basic axioms) on which the system is based.

      In this manner it becomes possible to explore new geometries, some of
      which may be found applicable to specific situations.
 

 

 
                                                                T.E. Batten, Amsterdam 1976


 

home person work texts links SAT