This semester, I have been helping out Jane with her wonderful Field Methods class in technical matters such as recording, uploading files onto the server and allowing students to securely and quickly download both .wav and .mp3 files. I took this course myself some years ago, and it was a great experience for me and the whole class, and many members of that class have continued on in their studies to do field research of their own, and I’m sure the Field Methods class was as much a help to their research as it was to mine.
\nBut this post is not about when I took the class. Instead, it’s about how I almost buggered up this semester’s class in what can best be described as a lesson in keeping backups of your recordings.
\n(Warning: Some computer nerd stuff follows after the fold.)<\/p>\n
\nThe course is being run in conjunction with Paradisec, which is where my helping hand comes in. We provided the equipment for the class to record their informants (two Karo Batak speakers), and provided space on our server for the recordings to sit. Eventually they will be archived in the larger Paradisec collection.
\nI always like to find ways of doing things quickly and simply using fairly simple programming. I’m not much of a programmer as such, but I know my way around bash, and have been using it to do most things automatically, such as moving things around and producing mp3 files of each recording.
\nThis week, the field methods students began their individual sessions with their informant, meaning that there are suddenly many more than just one recording per week; in fact there are closer to five or six per day, for two days per week. With this in mind, Jane suggested we organise the recordings into directories based on what day they were recorded. A very sound suggestion which I was happy to implement.
\nOf course, it would have been too easy to do it manually; so I tried to do it in a couple of lines of code. The first step was to take the names of the recordings (which are named in line with out specifications at Paradisec), and create directories based on those filenames such that they will contain all recordings on a given day. To take an example, we might have a list of recordings such as the following.<\/p>\n
The command I wrote would create two directories, called FM2-20100310 and FM2-20100317 (the command would also try to produce a directory for the last file, but it fails, since it already exists after being created for the second). Here was the code: This semester, I have been helping out Jane with her wonderful Field Methods class in technical matters such as recording, uploading files onto the server and allowing students to securely and quickly download both .wav and .mp3 files. I took this course myself some years ago, and it was a great experience for me and … Read more<\/a><\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[13,4],"tags":[],"class_list":["post-3886","post","type-post","status-publish","format-standard","hentry","category-experience","category-fieldwork"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.paradisec.org.au\/blog\/wp-json\/wp\/v2\/posts\/3886","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.paradisec.org.au\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.paradisec.org.au\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.paradisec.org.au\/blog\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.paradisec.org.au\/blog\/wp-json\/wp\/v2\/comments?post=3886"}],"version-history":[{"count":1,"href":"https:\/\/www.paradisec.org.au\/blog\/wp-json\/wp\/v2\/posts\/3886\/revisions"}],"predecessor-version":[{"id":4570,"href":"https:\/\/www.paradisec.org.au\/blog\/wp-json\/wp\/v2\/posts\/3886\/revisions\/4570"}],"wp:attachment":[{"href":"https:\/\/www.paradisec.org.au\/blog\/wp-json\/wp\/v2\/media?parent=3886"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.paradisec.org.au\/blog\/wp-json\/wp\/v2\/categories?post=3886"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.paradisec.org.au\/blog\/wp-json\/wp\/v2\/tags?post=3886"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n$ for x in *; do mkdir ${x%-*} ; done<\/code>
\nTranslation: for all files, make a directory of the same name, but strip off everything from the last dash (-).
\nThis worked fine, and despite some redundancy I had a bunch of directories, one for each day. The next step then was to move each file, like those above, into the directories that correspond to that day (which is always predictable from the filename). The code for this should have been:
\n$ for x in *; do mv $x ${x%-*}\/ ; done<\/code>
\nTranslation: For all files, move them to the directory which has the same name, but with everything from the last dash (-) stripped off.
\nHowever I missed the crucial forward-slash in the section ${x%-*}\/, meaning I had sent the files not to the directories of the same name, but to files<\/em> of that name.
\nNow, when you have many files of the same day, the output filename for this command is the same. So the way the command is run, it takes the first file, say FM2-20100420-01.wav, and moves it (which is synonymous with renaming it) to the file FM2-20100420. If there is a second file, let’s say FM2-20100420-02.wav, then it similarly moves it to FM2-20100420, thus overwriting what was there before.
\nAs I pointed out earlier, only this week did the class begin their individual sessions, so only this week was there more than a single recording in a given day. And therefore only recordings from this week were adversely affected (by which I mean deleted). The others were merely renamed.
\nLuckily, I realised what was going on by the fact that it was taking far too long to perform a mere move, and managed to stop it after only a couple of files got deleted. Even more luckily, especially as this saved my own skin, we have kept backups and the data is safe.
\nThe problem can be boiled down – computationally speaking – to a mere missing slash. But the real culprit here was my trying to be too clever by half.
\nSo let this be a lesson: Always, always<\/strong> keep backups. Especially if you are going to do any work on your recordings, even if you think it’s as mundane as simply moving them from one location to another.<\/p>\n","protected":false},"excerpt":{"rendered":"