What counts

If you are forced into evaluating scholarly work, consider the Linguistic Society of America‘s resolution on annotated language documentation materials (and see the RNLD list on this).

“Therefore the Linguistic Society of America supports the recognition of these materials as scholarly contributions to be given weight in the awarding of advanced degrees and in decisions on hiring, tenure, and promotion of faculty. It supports the development of appropriate means of review of such works so that their functionality, import, and scope can be assessed relative to other language resources and to more traditional publications.”

To which add grant applications [ update [ tx Claire!]- as places where track records get brutally evaluated by strangers ].
An article in an edited conference proceedings published by a scholarly vanity press counts for something. But it counts for NOTHING to have a carefully recorded, annotated, translated and archive corpus of recordings from say a year’s fieldwork, which would have taken at least a further year to create. The one creates something that most people will never read; the other creates something whose value is enduring. Result: scholarly vanity presses are springing up to cater for people’s need for publications. Possible solution: set up a scholarly vanity press for ‘publishing’ corpora. Call them books?

9 thoughts on “What counts”

  1. Jane I’m not sure what the “to which add grant applications” refers to here. I’m not sure that grant applications should count; grants clearly should (and do), but the applications themselves? Isn’t that a little like saying that rejected articles should also count for tenure?
    Lincom is essentially a vanity press and they publish text collections. I get the impression though that they are sufficiently much of a vanity press that publishing with them doesn’t count, since they are totally non-reviewed and not even cursorily edited.
    For what it’s worth (and I don’t think this is a widely held view, but I’ll say it anyway), I think we need to be really careful with arguments like “this took a year to produce”. That takes the argument away from the value of the materials and the thought and theory they take to produce, and pushes the argument towards the hours alone. That makes the argument more similar to the one we all see from students, that “I put a lot of time into this and I don’t think my grade reflects my effort”. (I don’t care how much time they put into it, it’s getting the right answer that counts, though that may be increasingly pre-modernist, absolutist and downright old-fashioned…)

  2. Oops! I was quite unclear. I MEANT to add evaluating someone’s track record in refereeing a grant application to the list of places where track record is evaluated.
    Yes there needs to be some kind of independent measure of corpora of annotated texts. But the value of many collections is in the presence of linked audio/video. So an online publisher would solve part of the problem. But many text collections can’t be published via online archives because of restrictions on distribution by the speakers. So perhaps the big digital archives could institute a rating system which could count as ‘publication’. Maybe using peer refereeing, together with a measure based on hours, detail, meta-data etc.
    And I agree about the need for caution over hours. But I am happy to accept that annotating a hitherto un-documented language with an unusual phonology and lots of irregularity takes more time than annotating a German dialect, and that there must be some way of giving weight to that.

  3. Clare and Jane: I agree that the issue of how to evaluate the scholarly merit of a documentary corpus is really interesting. In a field like Hittite, say, or Greek epigraphy, the people (archaeologists) who actually uncover the objects with primary language data on them don’t necessarily publish text corpora or even editions, but they publish excavation reports and the like. One strategy in our case might be to develop something (a journal? an online encyclopedia?) that has language-documentation counterparts of those excavation reports.
    Jane: There are West Germanic dialects with interesting phonological features, and (now I’m wearing my Indo-Europeanist hat) I’d like to encourage good documentary or text-corpus work on endangered and distinctive dialects of even well-known languages (or languages that are better known than most endangered ones).

  4. I couldn’t agree more about the value of having a collection of transcribed and linked material in West Germanic dialects. And I’d also agree that a collection with a very detailed narrow phonetic transcription would be a lot of work. What I was trying to find some measure for was assessing the amount of work involved in a particular corpus and assigning a value accordingly. Maybe this is not achievable – or not desirable. But there has to be a distinction between what could be done by an online transcription service, and what a linguist does in annotating and analysing texts and linking them to sound/video files

  5. It seems (to me) that presenting and analyzing texts, and at the same time presenting the underlying source of the material (an audio file, a linguist’s legacy manuscript, a clay tablet), is the traditional work of text editing. Text editions can be published, and getting linguists credit for doing text editions shouldn’t be all that different from what’s customary in universities. (Sometimes linguists don’t value that kind of work, but I think that’s a different problem.) It seems (again, to me) that the more serious problem is getting scholarly credit for doing the documentation work: the work that’s analogous to digging up new texts. What if you’ve created a great documentary resource for the next text editor who comes along? How do you get credit for that? (Or should you?)

  6. One possibility is a recognition not of the “digging up”, but of the fact that because “digging” is time-consuming, especially if done well, that has an effect on output of other types. I think that’s an important distinction. When I think of the time that goes into corpus creation and materials preparation, a lot of it is fairly basic formatting, cutting and pasting, etc. It doesn’t require much skill (beyond an ability to understand the materials, which is what makes it necessary for the linguist to do it and not a random undergrad or a “mechanical Turk”) but it’s a necessary evil, like assays or making reagents. Getting “credit” for it seems to me to make the job more about that, rather than that being the annoying and boring stuff we do to prep or postprocess the actual, interesting research. I guess the analogous item in theoretical economics would be how many notepads of algebra the person filled on the way to the published article.

  7. This sounds like a brilliant well-thought-out proposal for something we could all benefit from. I guess it did not get funded when you applied first time round. Are you applying again?

Here at Endangered Languages and Cultures, we fully welcome your opinion, questions and comments on any post, and all posts will have an active comments form. However if you have never commented before, your comment may take some time before it is approved. Subsequent comments from you should appear immediately. We will not edit any comments unless asked to, or unless there have been html coding errors, broken links, or formatting errors. We still reserve the right to censor any comment that the administrators deem to be unnecessarily derogatory or offensive, libellous or unhelpful, and we have an active spam filter that may reject your comment if it contains too many links or otherwise fits the description of spam. If this happens erroneously, email the author of the post and let them know. And note that given the huge amount of spam that all WordPress blogs receive on a daily basis (hundreds) it is not possible to sift through them all and find the ham. In addition to the above, we ask that you please observe the Gricean maxims: Be relevant That is, stay reasonably on topic. Be truthful This goes without saying; don’t give us any nonsense. Be concise Say as much as you need to without being unnecessarily long-winded. Be perspicuous This last one needs no explanation. We permit comments and trackbacks on our articles. Anyone may comment. Comments are subject to moderation, filtering, spell checking, editing, and removal without cause or justification. All comments are reviewed by comment spamming software and by the site administrators and may be removed without cause at any time. All information provided is volunteered by you. Any website address provided in the URL will be linked to from your name, if you wish to include such information. We do not collect and save information provided when commenting such as email address and will not use this information except where indicated. This site and its representatives will not be held responsible for errors in any comment submissions. Again, we repeat: We reserve all rights of refusal and deletion of any and all comments and trackbacks.

Leave a Comment