Monday, December 23rd, 2013
This article was originally written for and published at Philly 2600 on December 23rd, 2013. It has been posted here for safe keeping.
It’s rare that I get overwhelmed. I’m not talking about stress or anything like that. It’s rare that my senses get overwhelmed, specifically my sense of sight. This past Saturday, that sense became overloaded.
I’ve known Aaron for a little while now. We met online somehow in 2012, and while I don’t remember the exact details, I think he started following me on Twitter and things went on from there after I followed him back and we started replying to each other’s tweets. We quickly figured out that we lived pretty close to one another, which I found humorous considering we were both into archiving and preservation. Who would think that I’d be geographically this close to another person who idles in the #archiveteam IRC channel, online headquarters for the team dedicated to rescuing any and everything in the way of data? Aaron and I hit it off pretty well, and we eventually ended up meeting (somewhat unexpectedly) at Pumpcon 2013. Later, I ran into him again at the BSides Delaware conference and shortly thereafter he started coming to the Philly 2600 meetings which I’ve been frequenting for some time.
About two weeks ago, Aaron approached me via an online message and asked if I would like to go through some old computers at a local nonprofit he is on the Board of Directors for, NTR. NTR is in itself a fantastic organization which provides both refurbished computers (done in-house from donations) and hands-on computer training to low-income Philadelphia residents. If you are employed by or know a company in the area that is retiring their current fleet of workstations, consider donating the old machines to NTR. And, if they ultimately cannot use the machines, they will ensure that they are recycled in an environmentally safe fashion.
Aaron thought that I would be the right guy to help out. Being someone that preserves old technology, rescues it from unknown fate, and is a general enthusiast about it, I couldn’t resist the urge to come out and see what I could uncover. The details I got about what I was to do left a lot to my imagination. I got a location, we settled on a time, and I was told to wear clothes I wouldn’t mind getting dirty and bring a set of work gloves. Hardhats would be provided.
The dirt and grime never bother me. Just what I would be working with, I didn’t know. But, I was excited nonetheless and on the morning of Saturday I walked on over to NTR and met Aaron out front. The building we would go on to enter was the former site of the hackerspace The Hacktory before they moved to a larger location. The building itself is a big old warehouse that is much larger inside than it looks from the street. The parking lot to the side is encased with giant stone walls almost as high as the building itself and easily fits a dozen cars without having anybody blocked in. Aaron tells me that the building has also been declared a historical site, meaning they can’t do a lot of modification to it directly, but they do keep it nicely maintained.
As Aaron lifts one of the giant metal doors encased in the building’s western wall, I get my first look into NTR. He shows me bins of donated computer equipment: smaller stuff like peripherals lovingly stacked in re-purposed milk crates and small amounts of desktop computers stacked together up the side of the two-story wall. I get a tour of all the classrooms, a look into the computer thrift store they run out of the same building, and dozens of other rooms and hallways that wind around the giant space, separated by heavy opaque sliding doors. Eventually we make our way into the main computer storage area where there are pallets upon pallets of donated machines on giant shelves that Aaron points out to me with a flashlight. It’s dark in this part of the building.
We then go up to the second floor to see Stan, who is the Executive Director Emeritus of the organization, having initially been the Executive Director starting in 1980 and taken on the Emeritus title more recently. Stan himself is energetic and charismatic and goes on to tell me about how he set up a community information store on South Street in the 1970′s as we head down to where we came in to the building to the relatively new looking wooden steps that will lead to the area that Aaron and I will be looking through for the next few hours. Aaron later explains that much like me, Stan has been collecting and preserving technology and computer history, though he has been doing it for considerably longer. Some of his collection is also mixed in with the stuff we will be digging through.
I put on my gloves and snag a hardhat out of milk crate on a shelf by the stairs before Aaron and myself head up. The stairs are steep and don’t seem to be spaced consistently. You feel like you could fall down them easily but the railing is firm enough to keep you steady. As we make it to the top, I peer into the sea of computers which I will be acquainting myself with, lit by a pair of metal lamps that are clipped on to the wide beams of the underside of the roof – an afterthought in this 40×20 foot space.
A shot behind me after I made my way off the stairs
I quickly realize I can’t stand up all the way and have to hunch over, but that isn’t nearly as assaulting as the dust that comes out from seemingly everywhere and permeates through the air thick like smoke. Aaron walks slowly forward with his flashlight in hand and I follow close behind as he points out different areas of the space. We see newer stuff like a few Dell servers and stacks of Intel-based PCs at first but as we go further in we take more steps back in time. Aaron shines his light on a pile of all-in-one Macs before going further to the more interesting artifacts. On the left are some more modern machines, followed by boxes upon boxes of various documents, computers, and peripherals. I see Kaypros with Commodores with IBM clones and crazy displays for systems I can’t even fathom. There are tons of Macs, a few Mac clones, Apple ][s, and some old portable computers the size of suitcases. There are bags of electronics: half finished projects from decades before, muddled in with 8-bit personal computers, a pile of Sun workstations, and boxes of 5.25" floppy disks. On the right side are more Macs: G5s, G3s, a dozen classic Macs, some older desktops and a seemingly endless collection of obscure monitors and terminals to other systems. This is where we start.
A view of the left side
A claustrophobic shot of the beginnings of the right side
We navigate down the narrow path separating the space straight through the middle and get acquainted with the Mac area. We line up rows of milk crates and start digging, sorting along the way. Put the classic Macs here, put modems in this bin, mice in that bin, terminals over here, MIPS-based hardware over there. We sort and sort and sort, moving the heavy machines slowly as we work another path into the mess. The day was a cold one, but we quickly discarded our jackets as we carried hardware along the narrow aisle we carved out; we were warm enough simply moving back and forth, ducking beneath low hanging beams and swiveling around waist-high stacks that created our own personal obstacle course. As we went, we stopped to appreciate anything interesting we happened to find. Almost immediately we come across a monitor for a NeXTcube (though we didn’t find the cube itself) and we dug up other odd monitors and software packages and interesting little add-on boards that most people have probably long forget. We pooled our expertise and our energy and sorted in a long sprint.
After we cleared a new path
Cleared path continued
Aaron told me that a lot of this stuff will ultimately be cleared out. The newer stuff didn’t necessarily belong there and could be assimilated downstairs or recycled while the less valuable systems would be readily sold at their retail store. Some of the rarer pieces would be donated to museums or sold to enthusiasts and collectors who appreciate them to ensure their longevity. I hope when the time comes I might fit into this last group. The amount of history in this room is simply breathtaking.
View from the far corner
After a brief break, we pushed back against the section we were using for trash so we had more room to sort. Ultimately, we successfully cleared space more terminals and bins upon bins of manuals – hard copies are always under-appreciated. We then moved around, more slowly, to some of the more obscure hardware – testing a few things as we went. More time in this stretch was just spent digging as opposed to organizing. We wanted to see what was in some of the giant boxes at the bottoms of the stacks. We didn’t want to leave any stone unturned. Who knows what would be tucked away? We sorted through some IBM clones, found an Amiga 2500, a Wang Terminal, a Vector Monitor, a Silicon Graphics Indy, a whole mess of Kaypros and some more interesting items like a computer for those with disabilities and a strange keyboard or computer that neither of us could quite figure out. Down below us, people were trickling in for a computer class in one of the many rooms. “Who here has internet access at home?” I heard an instructor ask before I accidentally knocked over a PowerPC Mac. Hopefully they didn’t mind the noise.
Delta Data IV “Cherry.” Keyboard or 8-bit computer?
Stack of Altos 580′s on some Kaypros next to a Commodore 128
We finally succumbed to the tech and called it quits for the day. We got a good idea of what was up in the area and talked about the next steps which are likely to be inventorying and testing (though there can probably be some more organization in the meantime). The space itself serves as a fantastic time capsule and it is a breath of fresh air to know that some of the stuff in there is just in there – and in good condition. However, there is much to be done and many more hours to devote to make sure everything is handled properly.
As we rounded out the end of our excavation, we threw down the hardhats and unhanded the once-clean work gloves before walking around the corner for a cup of coffee. As we took our first steps away from the building, I felt a sense of accomplishment. We were archaeologists returning from our first day at an excavation. We uncovered some great finds, having fun along the way.
With any luck, I’ll be asked back. There’s a lot to go through and I can’t help but think that there’s more I can offer. Never before had I been able to lay my hands on some classic pieces of hardware that I had only read about, and it was quite an experience being able to put the pieces together.
Univac / Sperry Rand keyboard
“Age means nothing today,” Stan told me earlier that morning. “In this day and age, things are moving so fast.” I can’t say that I disagree, but I consider myself lucky to have the experience and knowledge under my belt when it comes to vintage computers.
And with any hope, I can keep expanding it.
A shot of the left side from out path in the Mac section
Another shot of the left side
Some newer Intel-based PCs
More of the Mac area
Newer computers tucked away
More Macs, pink note states that this Mac was the second produced
Sun workstations, Macs, Apples, old laptops
RadioShack diskettes. Think the warranty is still good?
Close-up of the Altos 580′s
A lone Kaypro II
A Tandy and a terminal
The Amiga 2500 and an Apple monitor
Unknown brand keyboard
Timex personal computer
Another Kaypro II and a Kaypro 10
Monday, August 6th, 2012
This article was originally written for and published at The New Tech on July 8th, 2012. It has been posted here for safe keeping.
So maybe you want to be an archivist but don’t know where to start. I don’t claim to be an expert, but I have learned a few things going down this path that I can share. Let’s break this up into two main sections: digital media and physical media. No matter what you are archiving, you should first pick out what you’re going to save. You don’t have to put too much consideration into this step. You can be passionate about a subject you wish to preserve, be helping a group of people, or doing it for the hell of it. Archiving is at it’s base both a way to ensure survival and a way to fill a hole left forgotten.
In the digital world, you’re going to be focused on downloading, storing, and uploading. Let’s start with downloading as you’re going to want to get your hands on some media. I started on a Windows computer, downloading directly with a browser. You can get your hands on some stuff simply with a DownThemAll plugin and a lot of free time. For streaming content I turn to the video downloader plugin for Firefox or Replay Media Catcher. This lets you pull videos from all your sites like Vimeo or Youtube. Sometimes things aren’t as easy as clicking a file to download it. You may have to use download sites, ftp servers, Bittorrent, or something less standard. You never know. Get to know how to use jdownloader, an ftp client like Filezilla, and a Bittorrent client like uTorrent. You might find a whole slew of content on some obscure site and you need to have the tools to dig it out. Moving on to more Unix-like systems, learn all you can about wget, curl, grep, and bash scripting. I’m not going to cover how to use all of these tools, but with a little practice there are few things you can’t get when you use them in tandem. You would be surprised how simple it is to whip up automated processes that do everything you want with just a few sophisticated commands. Also, be on the lookout for more specified tools. For example, I found a fantastic tool for downloading Youtube videos called youtubedl. If there is something out there to be downloaded, there is usually a tool for the job.
When you get the data, you’re going to need to hold it. I was originally downloading everything locally, and still like to keep local copies of data I retrieve. Always keep your data on at least two drives, and preferably buy your drives in pairs so you can easily stick with this rule. You can never have too much storage. I currently have 15TB locally just for storing archived media. When it comes to other storage mediums, I’m not easily swayed. Data tape is expensive to adopt, and cloud storage lacks stability. In earlier days, I had only an 80GB drive, so I would back up to DVD+Rs. A lot of people will tell you that your burned media will go bad after about 6-10 years, but I have yet to have a disc become unreadable. I will say that I’ve had a lot of luck with Verbatim discs. I would coaster many discs by other brands when burning, but have only ever had one Verbatim fail on me. Stick with what your budget is, the price of hard drives are only going down, but if you’re a kid on a budget a spindle of DVDs can help in a pinch. Also, keep an eye on solid state drives. While I have yet to adopt them, they are the new thing in mass storage, though the price is still a bit steep.
So now you want to share your data. You have many options to consider. I’ve been using The Internet Archive most recently to place files which should be saved. Depending on your content, this may not be the best option for you. A few of the techniques I mentioned before to download from can be good options for quick data dispersal. For example, setting up a torrent for your files can be done in minutes, and fast FTP/HTTP servers can be rented for however much money you want to spend. The main points here though are longevity and redundancy. You want your files up for a long time, and you want them to stay online somewhere if one server takes a tumble. While torrents alone are terrible for longevity, they are great at getting data out fast. Combine this with a server, or a data hosting/streaming service and you have some type of redundancy. Always make sure your data is accessible.
So now you might want also want to save physical media. This can be as easy or as difficult as you want it to be. Saving your physical media is best done by making it digital. The shelf life of digital data is only going up these days while physical media like paper or tape only degrades. While I’m a fan of my physical media, transferring it to a digital format is the best way to share it and keep it alive.
When dealing with publications or photographs, you can generally get good results digitizing them with a decent scanner. I happen to be a fan of Epsons, but anything around $100 these days should be able to give you decent quality scans. As always, read the reviews just to make sure. If you’re feeling a bit more crafty, you can try your hand at creating a book scanning set-up. This can easily copy all of your publications quickly. After you make your scans, you can perform any type of compression you wish, and even take the resulting image files to assemble a PDF.
Video can also be challenging to save. Whether it be VHS, Laserdisc, or even DVDs, things can get messy. When dealing with your older analog media, you can find devices to digitize them. For example, you can get a capture card like a Canopis ADVC, or any number of DVD recorders on the market. There are also a slew of other little gadgets to clean up the video along the digitizing process. After you get the video converted, you can compress the raw capture down with a codec such as h264 to make the file more manageable.
Audio can be viewed in a similar fashion to video. You can easily pipe a tape or record player into a receiver and feed the output to a nicer sound card. Here, audio can be captured using a program like Audacity and saved as a lossless file or compressed with something like Vorbis or MP3.
With something to drive you, a little know-how, and a lot of time, you can easily start archiving the media in your world. Though it may be daunting at first, you can easily build as you go. Start small, and end up saving big.
Friday, July 6th, 2012
This article was originally written for and published at The New Tech on June 8th, 2012. It was a collaboration between Moonlit and myself. Enjoy
– Famicoman –
I think I’ve always been an archivist. A vital ally in the digital world. I’m the guy that saves a file from six years ago and pulls it up when people wonder whatever happened to it. I’m the guy who is going to make sure you can still find The New Tech episodes in 20 years, whether anyone would want to or not.
Some might call me a hoarder. Technically, by definition, they are correct. But just like how the word “hacker” has been usurped and manipulated by mass media, so has this term. The word conjures up television-tinted images of people living in trash and debris. It isn’t always like that. Things I save are organized, studied, and shared with the world, not rotting away in some closed off building. Not sealed from the world. If anything, I save because these items may be important to someone else. I’m not always part of the equation.
One could argue that you’re born with an archivist instinct. My philosophy has always been that to be able to look forward, we must look back. Besides digital data, I collect physical artifacts of our technological past. You can learn a lot about Blu-ray by looking at Betamax. This resonates in all archiving. There will always be someone wanting to know how we got to where we are, and hopefully he isn’t left with puzzled faces.
My digital archiving habits started with the world of internet video. In the beginning, I was maxing out my DSL connection and throwing videos up on to Google Video. That later evolved to the IPTV Archive and ultimately my current efforts with archiving Revision3 and a wider range of digital content.
Archiving isn’t an easy task. It isn’t just plucking files off of a download page. It’s mastering wget. It’s manipulating URLs. It’s fighting tooth and nail with a server for weeks, months. It’s talking to people, some of whom don’t want to be talked to. It really stops becoming a hobby and starts being a mindset. You begin to look at things differently, communicate differently, prioritize differently.
When I started out with the IPTV Archive, things were simpler. I could just go download episodes from show sites and be on my way. Now, I get to sites that don’t want to be downloaded in their entirety, and are definitely not set up to be. For example, last year I worked on backing up portions of good.net. After a while, they’d lock me out of their servers and the only way to keep downloading was to get a new ip address or wait the block out. This year with Revision3, their CDN throttles me, which ultimately just means I’m going to be waiting longer for their files. For whatever reason, corporations are not fans of someone downloading their entire library of material. Some entities are set up with commercial content, meaning eyeballs are numbers. If you mirror their content, they don’t get as many viewers and less viewers mean less money. In this light, I’m an enemy. I’m a thief. More importantly, I’m a necessity. Without me and those like me, entire cultures could be snuffed out like a flame. Many already have. It’s a strange feeling when you’re contacted by a show creator asking if he could download his episodes from you.
Archiving someone’s digital work is a weird concept to get your head around. Think if you were approached and someone wanted a copy of your entire website. Every little detail becomes theirs to thumb through, spread to others, and replicate for years after you’ve brought the original down. It’s weird, but it’s necessary. When someone years down the road says, “Man, I wish I could watch some old Revision3,” I’ll be there to say, “Here is a copy of all their content. Ever. Enjoy.”
It would be wonderful if it was all as easy as hitting a button and someone’s site downloads for you, but it’s never that simple. Most websites are not designed to be cloned so readily. They lack internal organization. When you peel back the layers, you’d be surprised to see how clumsily some large sites are maintained and held together with rubber bands and paper clips. Out of convenience, we can pull up the Revision3 example again. So many episodes are mislabeled, so many links are dead, the formats for each episode can vary at will, and there are so many episodes and full shows that are just outright gone to the point that if you had no prior knowledge, you wouldn’t know them to have ever existed. It feels like someone ripping pages out of a book and passing it off as if nothing happened.
You have to be one part resources, one part nice guy, one part detective, one part historian and one part hacker. You have to learn about the missing files, you have to track them down, you have to communicate with others who may have them, you have to have the storage and bandwidth to get them, and you have to do it all no matter what is trying to stop you. You have to do all these things, be all these people, at the same time. Sometimes, you have to do it as quickly as possible.
After you gather everything, there is always the question of how to preserve it and disperse it. You have to keep the files up, and make sure they’ll stay up. More importantly, you need to make sure that people can get to them without jumping through hoops. I’ve tried everything on this front. Torrent sites, ftp drops, streaming services, etc. but have ultimately cemented my toolbox with archive.org. For the uninitiated, the Internet Archive is a non-profit digital library offering permanent data storage. It’s big, and it’s growing every day. Anyone can upload content provided it’s licensed to be distributed openly. It makes things easy when I can be bringing things in through the front door, and flipping them right out the back to archive.org.
Digital archiving is a brutal but rewarding process that most people don’t see on the front lines. The next time you’re going to put something up online, take a minute to think about it. Your files are going to live much longer than you could imagine. You might as well make it easy for them to.
– Moonlit –
I’ve been a wannabe archivist for some time, but through a mixture of altruistic and less altruistic means, which just so happen to coincide.
On one hand I can’t bear the thought that there is so much recent history that may be, or in some cases already is, needlessly lost forever. Whether it be hardware, software or media, much of what is produced today has no vision for the future, it’s created, it’s used and, ultimately, it’s destined to be lost to whatever forces may eventually whittle its existence down to extinction. Failed storage media, the thought that “if I delete it, somebody else will still have it” or even just plain old waning interest in a flash in the pan which is no longer relevant tomorrow.
On the other hand I find it somewhat distressing that the content I grew up with, much of which came from TV rather than the internet, is very difficult to find. It’s just that little bit too old to have been swept up by a thousand torrent sites or archived to the ever expanding YouTube. It appears to me to exist in a narrow void between content old and popular enough to have made its way to public release via VHS or DVD as a nostalgia trip for the previous generations and the modern piracy scene, who will capture and upload almost anything as pristine digital clones of the broadcast content we enjoy.
Luckily, the two often overlap, so one can be the driving inspiration to accomplish both. But as long as the end result is shared, I don’t view the selfishness of the latter to be a problem. In fact it could very well be a boon, because if everybody was selfish enough to demand copies of the content they thought they’d lost, it means that content still exists, and given that everybody likes different things, meshing all that together would create a patchwork of content from that point in time.
Now, I’ve erred somewhat on the side of piracy so far, but I don’t mean to imply that I’m only interested in commercial media, or indeed in breaking the law. Before moving on though, I’d like to say that I think it’s a collossal shame that in order to capture and preserve certain parts of their lives, we often have to resort to methods which might seem unsavoury to those who disseminate that content. I don’t think it’s unreasonable to suggest that there are indeed large archives these days maintained by large media producers and broadcasters, yet those of us the content was created to be viewed by have no access. Whether that be through music or video clip copyright and licensing issues, laziness or cost, it’s still a great loss to us, and will continue to be until such a time that the content is opened up. This history should not suffer for the sake of a few contracts and a slew of many-digit bank balances. Please, somehow, let this content see the light of day again.
Whew. Got a little bit heavy there. User-created content, there’s a good place to jump to. Podcasts and video podcasts exploded in the mid-2000s along with the proliferation of high speed broadband and cheap consumer cameras. The trouble is, many of those shows had small numbers of fans who, along with the creators themselves, have moved on and left behind their content. This is an important chunk of internet history to me, it got me involved in a large percentage of what I do and who I speak to every day. That’s why I tried my best to help Famicoman build the IPTV Archive when we originally began trying to preserve this stuff. With my pitiful upload speeds and meagre hard drive space, which was frustrating enough, I helped transcode and re-host piles of videos. Those videos were then uploaded to DivX’s Stage6 video hosting site, all neatly encoded in DivX format, with their own special DivX player plugin. Then they took the service down. After countless weeks of pulling down videos, transcoding where necessary, uploading back to Stage6, straining my resources as I went, it was all for naught. Once bitten, twice shy, as they say, and since then I’ve been very wary of trying to do it again, but I’m slowly getting back on the horse. Lesson learned: redundancy. Redundancy and backups. Everywhere. Never rely on any single service to host this kind of stuff, it might be gone tomorrow.
Things get a bit weird somewhere in the middle of those two areas of content, though, with companies like Revision3. They began as a show, or later a couple of shows, which very much fit into the user created content model, a couple of guys with a camera drinking and talking out of their arses for 20-30 minutes. But then it changed. It became the Revision3 we have today, the corporate ad-driven sludge that could very well have been taken direct from the TV and uploaded wholesale to the internet. I’m not against making a profit on content, but stop sucking the soul out of it, it feels like it’s hurting the product. But I’m not here to rag on content creators, my point here is that no matter how poor, tasteless or boring I believe the content or its presentation to be, it still deserves to be archived. What’s crap for me might be gold to somebody else, and it’s not my job to curate history in the making. If I even began to try I would doubtlessly decide that something which later turned out to be pivotal in the future was actually the naffest thing to ever grace a visual display. I believe Jason Scott made a similar point about the preservation of GeoCities. Yes, it might be full of weakly written, poorly laid out, eye-damaging animated horribleness, but it’s historical weakly written, poorly laid out, eye-damaging animated horribleness. It’s a snapshot of what the internet was at that time, and as such it should not be forgotten. So go forth and grab it, grab it all, because as hard as it might be to believe, one day it will all be gone.
Saturday, December 10th, 2011
I’m pretty sure I hinted that I like to digitalize old formats. I’m that guy you see digging through bins of VHS tapes at yard sales, looking to find that one piece of gold that I haven’t seen and probably won’t find any other way.
You might not know just how into this stuff I am. I didn’t really know I was, but after years of accumulating relevant equipment, it starts to add up. I used to have everything sitting around in various piles. I’d keep some stuff up by the television in my bedroom to do simple transfers, and some of the bigger stuff downstairs where I couldn’t trip over it.
I recently decided to consolidate more, and bring most things to a dedicated area where I could do transfers.
I present the wall.
To briefly go over what we see here: The top row has some video enhancers, an audio enhancer, power station, and a DVD recorder. Next row down has some more professional video enhancers, a detailer, some boxes for stabilization and a full frame time base corrector / freeze. The next row has two Commodore 1702 monitors. 4th row has an SVHS deck and a laserdisc player on the left, and two editing VCRs with a Betamax deck on the right. The next row has 5 laserdisc players and a VHS duplicator. The last row has all my CEDs.
This doesn’t represent all of my gear (I got much more of this stuff on the other side of the room), but there is a pretty nice portion here for both storage and transferring. I can quickly wire up any format and start converting in a matter of minutes.
It has become something as a monolith to antiquated technology. I’m quite happy with it.