Tuesday, August 20th, 2013
Every Summer I speculate that I’m going to have an unbelievable amount free time. It will always be so fantastic and freeing. I’ll be done school, working a stress-free job, and there will be so much unscheduled time that I’ll just get bored and come up with hundreds of new tasks for myself.
This never happens.
Well, the having-free-time-thing never happens but I do take on new activities anyway. After enough time, I end up with a bunch of things I’ve been meaning to do, and work on them impulsively at sporadic intervals. Everything moves forward, slow and steady, but in an agonizing and chaotic fashion.
I do make time for my projects, but the available time is fluctuating as the years go by. When I started these projects all I had was time and energy, but no money. Now, I seem to find myself with a modest amount of money and energy, but no time. Eventually, I’m doomed to have time and money, but no energy. This is the vicious cycle, and here I find myself in the second stage.
Without organization, every project falls on its face. I’m a big proponent of organization, especially when I have so much going on. After a while, you just need to keep track and work smarter (or risk meeting some men who want to put you in a straitjacket). Below, I’ve outlined (to the best of my ability) the various projects I’m working on, and where they need to go next. Hopefully this not only helps me stay on track but also gives you something to yell at me about the next time you see me.
I’ve actually had most of an episode filmed for a long time by this point. The only problem that I faced was the audio cut out at the end of one of the shots. After I redo it, which I wanted to do anyway, the footage should be mostly set to go into editing. Additionally, I’d like to film a little skit for the intro if I can manage it. Editing usually doesn’t take a whole lot of time, though I do want to try out some new software and I have to cut a brand new introduction. High definition video also proves to be more of a hassle and take some more (read unplanned) time.
This one is going along pretty well, especially recently. On the scanning side of things, I have plenty of stuff coming in but not a lot going up. The scanner I have is awful when it comes down to conducting magazine scans and I’ll have to look for something beefier before going full tilt on my library. As an aside, I’ve more or less created the most complete wiki of hacker magazines complete with information on them as far as I can tell. With my current rig, I can pump out some more Blacklisted! 411 issues without much hesitation.
Going after Revision3 has slowed a little, but I can get back into it with some one-liners soon. Getting to other odds and ends comes and goes as I find them. The only section that could have hours poured into it is the hacker con category. The videos I find not only have different ways of being obtained but also get updated with a new crop annually, so everything is constantly in flux. I’m trying to hunt down some of the more difficult stuff as well as fill in actual information about the conferences. If you want to help out, please do.
This one is more or less dead due to lack of interest. While it was cool having a collaboration site for retro tech, it lost its luster after a few months. I considered turning TechTat into an audio podcast but I’m not sure how that would turn out. I’m certain I can find some use for the concept.
ChannelEM keeps trucking on, but is prone to frequent crashing. It does seem to get more stable after software updates, but still ultimately hangs. I want to take a look at the scripting done to run the station and see if I can put in any fail-safes to stop the crashing. CEM also needs a rotation update with any new episodes. Further, the idea of getting new shows to join up is a bit fruitless now, but the site does well as it stands. For no real reason at all, I’d like to see if I can add on to the existing scripts and create a JSON API with scheduling information.
Moonlit has also been working on some very interesting video projects that I’d like to integrate which would completely change the look and feel of both the site and the content.
More or less in a standstill. The stability fluctuates and there isn’t that much going on there anyway. The IRC server is really only kept up if Ethan, Pat, and myself need a place to chat. For a network that has been off-and-on for 6-7 years, we have empty periods like this all the time. I’d like to just keep it up if I can.
The IPTV Archive
More or less in waiting. I put up a hefty amount of content, and then ultimately mirrored it to Internet Archive where it can live forever. If I had the time, I’d spend it doing more detective work for the missing shows- there is always more detective work to do. There are probably a half dozen more smaller shows I could throw up at some point but nothing too pressing.
Additionally, when I started the site I used Blip because it had (arguably) the best quality at the time. Now, YouTube has eclipsed it. There was a bit of panic a few months back about some Blip channels being closed down for no reason and I have to entertain the idea that this could happen to me. If that happens, the whole library would likely need to be moved to YouTube. A big move, but likely a nice one for the content.
Moreover, I’ve also considered moving the content over to Anarchivism as it would be a much more flexible platform.
House Keeping and Solo Projects
I enjoy writing and I’d to do more of it. Besides just being more active here, I’d like to get back into writing for other outlets. I’m thinking of more for The New Tech, and another for my local 2600 group. I’ve also been playing around with Medium (I like the concept but it still might be pretentious dribble) and would like to publish another article through it. I’m looking into 2-3 print publications as well if I can come up with the right topics and go into those pieces with the right energy.
Aside from my web work, I have a bunch of little, lower-profile things going on that I need to get out of the way.
I recently got a display for my Apple G5, so I can let it run as a capture PC for video transfers. I already have an ADVC box hooked up and the machine captures great… but it needs a monitor hooked up to run. Then, I can do more video transfers which can ultimately pop up in other places (Maybe a found footage section on Anarchivism).
I want to set up a dedicated headless Linux server for staging web projects amongst other things. I might also have it just run wget scripts all day or some custom web crawlers or who knows what else.
I have an old cocktail arcade cabinet that needs some love. If the original electronics are beyond repair, it would be nice to outfit the cab with new hardware and set up a MAME machine.
More Raspberry Pi projects would be nice. I like having the Incredible Pi set up as a PBX but I feel like I could do more with it. I have another Pi set up as a media center that I use often. I’m currently on setting up a Bitcoin mining rig with another and still have many more ideas. Raspberry Pi cluster? Telnet BBS? BBS hooked into the PBX? The possibilities are endless.
Paranoia kicks in with regard to my data. I have a dozen or so terabytes worth and I need to clean data off of old drives, sort it, duplicate it, and duplicate the data that’s already there. To make matters worse, I’m constantly downloading more.
A CJDNS Meshnet node has also been in the works for a long time. I tried to set up my first one on a PogoPlug and while I eventually got the software to compile, I couldn’t connect to anybody. It may be time for another try, and possibly on a “normal” box before adapting it to the PogoPlug.
And the list goes on.
There’s a lot of things here- a hell of a lot of things. I’d be lying if I said that there weren’t going to be even more. Hopefully, as I now have a nice little outline, I’ll be able to zero-in my focus and get some work done.
In the meantime, sit back and enjoy the show.
Tuesday, April 23rd, 2013
Yeah, I stole the title from the Halloween series. It seemed fitting.
So here we are about a year later. Guess what? Revision3 has killed off a few more shows, started a bunch more, and redesigned their site.
A few days ago, Moonlit and I got into an interesting Twitter conversation with Revision3, the results of which can be found here and here. Basically, with their site redesign they lost a bunch of shows which they then regained after we brought it up. They also claimed to fix some dead links, so some possible good news there. On the negative, they did say that some shows have been removed purposefully. Thankfully I believe I have a copy of everything they’ve taken down to date. On the whole, it felt as though they danced around issues I brought up, but at this point I find it unsurprising. It’s probably a good thing they didn’t Google me while we were talking.
I checked my download script from last summer, and sure enough it didn’t work. Upon doing a few trials, it’s a simple fix to change “small” to “medium.” So, I went ahead and updated it. This now works perfectly again (woohoo!).
I went through the wiki page and added in the shows that they had cancelled while I was out of the loop. I’m just going off of their “Archive Shows” page, which may be incomplete, but I don’t know enough about this age of Revision3 to tell you if something is missing or not. It doesn’t look too much as if they throw out shows altogether these days.
I count 10 more dead shows.
On a similar topic, I’m also finally getting work done with Hack College (One of the baby Rev3 Beta shows) and am in the process of uploading it. Let’s walk through how I did this finally because you might be curious.
I know that they have the series up on http://blip.tv/hackcollege. This is good for starters. I also know that youtube-dl supports Blip.tv. Excellent. After a little trial and error, I settled on this command:
youtube-dl -c -i -t http://blip.tv/hackcollege
This downloads all the videos from the hackcollege account, puts titles in the file names, continues incomplete downloads, and skips errors. I only added that last part because one video gave me an error (I eventually just downloaded it manually). Then, I forgot I wanted descriptions for each video, so I ran this:
youtube-dl -i -c -t –write-info-json http://blip.tv/hackcollege
See how easy that was? JSON descriptions in just a few seconds. Okay. A few people still know a little trick for getting RSS from Blip.tv: just add “/rss” after the account url. You can go ahead and load it yourself at http://blip.tv/hackcollege/rss. Now, if you’ve been downloading along, you might notice that youtube-dl only snags the .m4v files and there are these beautiful giant .mov files in the RSS feed. Wouldn’t it be nice to grab these? Unfortunately, you can’t fine-tune youtube-dl for these just yet (maybe if I hunt around in the source code I can set this up, but I honestly didn’t think of that until right now). How do we get these files? I came up with this one-liner:
curl http://blip.tv/hackcollege/rss | grep -o ‘http.*mov’ | sort | uniq > out.txt
Pretty self explanatory. Grab the RSS, filter for .mov links, and sort it to get rid of duplicates. Then, save it to a text file. You might notice that the RSS feed doesn’t contain links for every video. There really isn’t anything we can do about that, but it does appear that the .mov files were a semi-recent addition to these videos and earlier videos most likely don’t have the option. I’d say it’s a pretty safe bet that we’ve got them. Next, we will want to download those links. You can probably pipe that one-liner into wget, but I wasn’t ready to download when I wrote it, so I saved the text for later. Here’s how I eventually used it with wget:
wget –user-agent=”iTunes/10.6.1″ -c -i out.txt
It appears that Blip.tv white-lists your user-agent, so it knows to not allow wget, or probably a slew of other bots and/or browsers. You can find this out if you can download a file in your browser but running something automated on the site results in redirects and file fragments. There’s always a workaround. If you trick it into thinking you are iTunes, you not only get access to the files, but also get them faster than if you just manually downloaded from your browser (throttle-free!). Very nice. So after feeding the text file into wget, the .mov files download rather quickly. You don’t get the nice fancy file names like with youtube-dl, but you do get the files.
So there. Only took an hour or two to work everything out.
Well, as you can see, I’m finishing up a few parts of SaveRev3. Unfortunately, there is still work to be done and the list keeps on growing.
Here’s to another summer of hard drives and bandwidth. It’s going to be a hot one.
Friday, February 8th, 2013
I have many projects. Too many, one might argue. Either way, they exist and I enjoy doing them.
For a long time, I’ve had something of a collection of magazines that I usually refrain from talking about simply because it doesn’t come up a lot in normal conversation. A few months ago when it was announced that Nintendo Power was halting production, someone told me that the cover of the last issue was a throwback to the very first issue from 1988. I was asked if I had the first issue (people tend to wonder just how much old stuff I have) and I do. Here’s a picture of it.
Nintendo Power #1
What you don’t see in this picture is the rest of my magazines. I have a lot. Hundreds. Most of them are video game magazines from the 1990′s and I’ve been accumulating them for over ten years. That isn’t to say that these sum up my entire collection. I have an almost complete run of 2600, six or seven years of Wired, a few dozen issues of MAD Magazine from the 1970′s bundled away, 10 or so issues of High Times from the late 1970′s and early 1980′s, and a few years of some more modern things. Besides those, I have a few other random magazines here and there and most likely some I’ve forgotten.
While I have a few current subscriptions, I’ve recently re-opened my magazine obsession. Why now? I don’t really know, but it was bound to happen. Every once in a while you get one of those “I should really do that, wouldn’t that be great?” ideas and they really start to stack up. One or two of those ideas end up toppling off the pile sooner or later and you just run with them. This particular idea started with Blacklisted! 411.
If you haven’t heard of Blacklisted! 411, I don’t hold it against you. If you know what 2600: The Hacker Quarterly is, then think of Blacklisted! 411 as a lower budget version of that. If you don’t know what 2600 is, it’s the most popular and longest running independent print hacker magazine. Blacklisted! has something of an interesting past. There are a lot of politics involving the magazine that are still something of a hot issue even for people today. There’s no doubt that it left a sour taste in the mouths of many. To briefly go through things, the zine started as a cheap black and white publication in the early 90′s. Initially monthly, the magazine switched to a quarterly release schedule to allow for more articles per issue (mirroring 2600 in this regard). Many criticized the quality of the articles and the publication in general, but it had a loyal group of fans and writers. In the mid 1990′s, the magazine up and disappeared (angering many) and reappeared in the early 2000′s. Throughout the life of Blacklisted!, a lot of people claim to have been treated unfairly by it and promised compensation for their articles which they never received I wasn’t there, and I don’t know all the details for sure. Defending nor attacking the magazine are not my goals either way.
Issues of 2600
For as long as I had known about 2600, I had also known about Blacklisted!. While I could easily get back issues of 2600 through their website, Blacklisted! was far more elusive as it went out of print. I was less likely to come across old issues out at book sales or flea markets when compared to something more popular like Wired. So, I forgot about it for a while and chalked it up to a boat that I had missed.
Fast forward to now. I’ve decided to take it upon myself to start gobbling up every issue of Blacklisted! 411 ever produced. Normally when you see someone take on a pie-in-the-sky task like collecting all of something from scratch you dismiss them with an “oh, that’s nice” and pat them on the head while taking bets on how quickly they tire of the project and go home. I already know it’s not something that will happen overnight, and will probably take years if I’m ever able to complete it at all. It’s a bit of a turn-key project either way, so it’s not much of a hassle. Initially, I set up some aggregation online to see if any issues go up for sale, at most I might dig through a few more bins at the punk rock flea market. It’s something of a slow burn.
As I started doing research on Blacklisted! I came upon a few other hacker or hacker-related magazines that went into print. For example, I discovered Mondo 2000 (and its other incarnations), bOING bOING, THUD, Grey Areas, Binary Revolution, and more. These were also low-number interdependent physical magazines that lived a short life of usually fewer than 20 issues. So, I expanded my scope. If I can find them for the right price, I’ll snatch these up as well. Are there more out there? Probably (And please, let me know what I missed). I can’t get to everything, but I have a pretty good idea of what print zines we had just by asking around.
THUD & Binary Revolution
You may raise the issue of me going after physical magazines exclusively. Where’s the love for the electronic zines? While I do have a fondness for ezines, I don’t consider them nearly as endangered a species as the print-only zines. While an electronic zine may have been copied hundreds of thousands of times with little effort, when a physical magazine goes out of print it can only slip further into obscurity. Some copies get mistreated and trashed, while others are packed away and forgotten. These are the ones I want to save. Right now at least.
So the next logical question is what am I doing with all of these magazines? While I admit that I do get a nice warm, fuzzy feeling from physical magazines, I have bigger plans than simple self-satisfaction. Scanning is the name of the game. I’m currently in the process of scanning in all these old issues I’ve already found, compiling each issue into a single document, and uploading the documents online to share with everyone. Through this whole scanning process, I’ve already learned a lot. Enough to write something on it actually, but it would fare better as its own article. My scanning workflow works well enough to actually yield results, which you can check out here and here. If you want to check out my overall progress on how I’m doing with all the zines I hope to find, you can visit this page. You might notice that in some cases, I’ve found magazines already scanned by people. These are few and far between, but save me a little work considering they are usually of good quality.
While my scanner might be slow and I might be busy, I’m happy to say that the wheels are in motion. Things would probably move a little faster if I had a more portable scanner, but for the time being I’m keeping things slow and steady. That all said, if you have some of these magazines and feel like donating to the cause, I’ll serve as a home for your wayward magazines (and I’m probably a decent alternative to the trash if anything). If you feel like scanning, you can contribute that way as well. The Anarchivism wiki linked above is editable if you create an account.
So as I’m picking up older magazines, I’m also starting to focus on newer ones. Consider something like Bitcoin Magazine. An independent publication about a decentralized digital currency? Who knows how much longer this will stick around. It’s important to apply a little foresight for things like this. Otherwise, who knows what you’ll be able to get your hands on down the line. Luckily, many current publications have bridged the digital divide and offer both physical and electronic copies. Other magazines are now entirely based online. Still, there are those holdouts that are only available on paper. These are what I’m after. These are what I want to save before time runs out.
Friday, November 23rd, 2012
It’s been a while since I’ve done one of these.
My progress on archiving had been stalled for a little bit. I got about half way through the Revision3 Beta shows, and then had other things that demanded my time. Working through the past week I’m down to just one more show which I am trying to download as I type. Surprise, sometimes downloads don’t go the way you want them to. Focusing on Revision3 Beta shows, I believe I’ve already touched on how these shows are hosted for streaming on Viddler instead of the standard BitGravity download CDN. But why you may ask? With those unfamiliar with the concept of Revision3 Beta, I don’t hold anything against you. It’s another one of those little failed projects by Revision3 that you’d be lucky to find any information on these days. Here is an archived page of the line up. These were little independent series that were “talent-farmed” (quoting Wikipedia on this one) with hopes that they’d one day make it to a full-on Revision3 show. A pretty awesome idea if you ask me, and something that had a lot of potential. The project was apparently halted because of a lack of funding, but if you watch any of these shows you wonder what the hell that could mean. The shows don’t seem to be getting any money, running Viddler accounts can’t be too expensive, and basic web hosting is pretty cheap. I’d wonder if someone just didn’t want it around anymore. Even though Revision3 shut down the Beta project, many of these shows went own to produce more episodes outside of the Revision3 banner, but most ended up with a short lifespan anyway. Most of the shows continued to upload to Viddler and these episodes basically have to be downloaded manually. This can be a pain if there are many of them. Thankfully, a lot of these shows uploaded their back catalog of episodes on other sites such as Youtube, Vimeo, or Blip.tv. Using youtube-dl, downloading these videos from more popular sites can be automated for the most part, making things much easier. It would still be nice if the tool had Viddler support, but Viddler isn’t a popular platform and beggars can’t be choosers.
Going back to regular old Revision3 shows, Unboxing Porn has finally been moved to the Archived page after being out of production for quite some time. On top of that, Ask Jay and Epic Meal Time are now moved over as well. So, I did what anyone would imagine and performed a download sweep of those shows. As usual, I found some numbering errors. Epic Meal Time goes out of sequence for two episodes, and has a completely messed up episode feed. Fortunately, I was able to pull all of the episodes they put out under Revision3 without any considerable hunting. Ask Jay turned out to be perfect, which is always helpful.
While I was originally worried about the state of Revision3 after the Discovery deal, I’m surprised by what I see today. While I thought many shows were going to be cancelled, the network is actually growing considerably. I’d estimate that the number of shows currently being produced has almost doubled since the start of the summer. I can’t help but wonder if this is a measured approach, or if they’re just throwing shows against a wall to see what sticks. Either way, I’ll be sure to follow up and see what they axe.
The more shows they end up making, the more I’ll end up preserving.
Monday, June 25th, 2012
So we’ve come a long way. All of the archived shows have their episodes up, and some of them have episodes up that weren’t produced by Revision3. This is a pretty big point in the project, so breathe the fresh air while you can. A few shows still need show notes (Did I mention that we have almost all the show notes as well?) but things are coming along nicely, though a bit slower.
On top of archiving the archived shows, we have also taken care of some of the shows on hiatus. Moonlit and myself have also begun working on Revision3 Beta shows. Some of the Beta shows are missing episodes, and other have them scattered all over different services, but we are slowly gathering and compiling them. All of the shows were originally hosted on Viddler, and using some URL generation, it is fairly easy to make download links from the videos. However, some of these generated links end up dead which means the streams have to be downloaded manually. It’s a time consuming task downloading and checking, but definitely doable.
As for currently produced shows, the best thing to do might be to upload episodes in year chunks. Either way, it is probably a good idea to start downloading these shows now, just in case the worst is to happen.
That said, I’m currently dividing my archiving a bit to a few more projects, but saving Revision3 is still on the list.
We’ve almost finished getting the shows most have forgotten about, but we will still fight to get the others before they can be forgotten.
Monday, June 11th, 2012
Shorter update today, let’s just cover the main points.
For starters, Moonlit and myself did an article for The New Tech about archiving. It’s a cool read if you have the time, and I’ll eventually put it up here.
In the past week, Moonlit found a fantastic way of downloading episode descriptions en masse with a cool tool called Outwit Hub. This automates the whole task of getting descriptions. After these are pulled, I check them and add show descriptions before uploading to archive.org. I’ve done a handful of shows so far, and more are on their way to being completed. I also have started downloading the last two shows on the archive list, which are more of a pain to do but at least I’m getting them done.
As I said earlier, my next phase will be archiving Reivision3 Beta shows. To do this, I’m going to need a mass Viddler download tool to avoid going insane. Just something that will pull all the videos from a profile. If anyone has any ideas, send me a tweet, come chat, or leave a comment.
Revision3 Beta Logo
Others will probably want to go right to archiving shows currently being produced. I’m still trying to figure out the best way to do this. For now it’s probably a good idea to download first, and ask questions later.
As always, I’ll keep you updated and feel free to stop by if you want to help out or have any ideas!
See you soon.
Tuesday, June 5th, 2012
Well, things are going pretty smoothly now. If you look at the page, most of the content is up. That is, most of the shows that have been put in the Revision3 “archive” have been downloaded, sorted, inspected, and re-uploaded to archive.org.
It’s taken a long time.
This is day 25 of me downloading and uploading. All day. Every day.
What have I learned? Revision3 cannot take care of their content. I’ll give them a little credit. I have a list here of notes from when I started this project two years ago. It’s a short list of little issues I encountered, some of which have been fixed. This sort of surprised me. For example, there was originally a missing episode of popSiren. Missing as in not in the episode list, but it’s back now. I can’t say the same for most shows that are now in the archive. I’d say that about half the shows I’ve downloaded have one problem or another whether it be missing content, mislabeled episodes, or some other issue that you couldn’t even wrap your head around.
All the issues I stated in the last articles, they just get worse. It would be a waste of article to address them all, so if you’re really curious just click the link to the wiki page above and look through the notes section of the table. There are a few that I may have fixed and not mentioned, but most of the issues are there in readable little sentence fragments.
So where do we go from here?
I’m currently trying to scrape together the last few shows. After this, there will be a *complete* mirror of all the Revision3 archived content. I use that world complete to mean everything they offer. There are shows with holes, and I note them. I’ve also investigated some of them and found that there is true mislabeling, but some I can’t tell for sure. After I get all of this sorted through some detective work, I have to go into descriptions. This will take hours in itself (some help would be nice) so that all the content has episode descriptions. I didn’t do this initially because I wanted to make sure I had all the content. You can scrape episode descriptions from other places if things go bad, but you usually can’t find two dozen shows when they just disappear overnight.
let’s cause this descriptions bit Phase II.
Phase III will be getting other content. Yes, we got the archived shows, but Revision3 still produces content daily. This needs to be grabbed too, but questions come up. How do you archive a show that still makes video? Local copies until it dies? Upload it yearly? I’m not sure yet and am open to ideas. We must also go into Revision3 Beta, the website for which is now gone. This is where I will be focusing many of my efforts to try to snag what I can before the internet forgets it completely. Lastly, I want to expand to shows that Revision3 produced in part, but not fully. For example, I have complete runs of The Game Show and Epic Fu, both of which now appear dead. There are more shows like this, and more still going on that need to be monitored carefully. We don’t want anything disappearing into the aether.
Again, I ask for your help.
We need people. We need people for a variety of tasks large and small. What can you do to help? Download episodes. Gather descriptions. Come talk to us. Give us ideas and help us think in ways we haven’t yet. More than anything, spread the word. You might have no interest in any of this, but if you think the project holds any merit, donate a re-tweet. If you have a podcast or blog, interview me. If just one of your hundreds of internet followers is a fan, that’s one more person who might want to help be a part of this massive (and often crazy) project.
This is only the beginning. We’re going to need a bigger boat.
Tuesday, May 29th, 2012
So here were are with Day 19, and boy has the project come a long way. To the handful of people helping so far, I want to give thanks. From those who download, to those who scavenge, to those who re-tweet: it wouldn’t be possible otherwise. Having said that, it seems that every day, with every new show, problems arise.
Allow me to show you just what I mean.
Let’s talk first about Web Drifter. In the beginning, there were four episodes of the series before it stagnated. Then, the episodes were wiped from the site completely. Just gone. Then two more episodes were produced: episodes 2 and 3. But wait, these episode numbers already existed didn’t they? Yep, Revision3 put up new episodes 2 and 3 as promotionals working off of “episode 1″ which I guess was their way of saying there used to be more episodes. Do you follow? I sure as hell cannot. So, after some creative URL games, I managed to recover all the episodes they stripped from the site for whatever reason.
Now, what about Diggcetera. If you go to the site page, you might see that the first 14 episodes don’t have download links. On top of that, the streaming versions don’t work at all anymore. So what the hell do we do? Luckily, Moonlit had a bit of time to do some detective work and found a Youtube channel and Vimeo profile for what appears to be Diggcetera videos. From here, going by page name on Revsion3, we could relate that back to the episodes on Vimeo and get them all sorted. Now, here’s something interesting Moonlit discovered when trying to determine if the order of episodes was correct: If you look at the thumbnails, though they are correct for the episode they are numbered and labeled for the wrong ones. Someone labeled screenshots with incorrect episode name and number when making the pages, but not consistently. Why would anyone do that? An intern could work through most of these issues in a half an hour.
Let’s move forward to something “slightly” newer: The Digg Reel. When I downloaded this two years ago, all the episodes were up and linked, so downloading was pretty easy. However, now it appears episodes 155, 151, 149, 145, 141, 131, 113, 102, and 14 are gone from the site. As in, the pages have just vanished. I still have the episodes from back then locally, so I know the titles, and trying to retrieve the pages pulls up nothing. So what happened to the pages? Who removed them well after the series ended? Luckily, the episodes are still on their servers, so redownloading is a breeze. Something else bizarre with this show is that it had the first 28 episodes in hd720p30, but episodes 29-78 are only available in standard hd. It is as though they didn’t know how to work a new camera for a bit or something.
Bytejacker is an example of a show that was independent and later picked up by Revision3. Not only did Revision3 start the series at episode 1 from when they started carrying it, but they also create gaps in the numbering system. So now, archiving is seen as a bit of a nightmare. We’re at episode one from Revision3, but it’s further down the line when you look at the series as a whole. And the gaps don’t help in the slightest. These gaps spring up in several shows, and it’s often difficult to tell when the episodes existed and were removed, or if they never existed at all.
Let’s take a break from talking about Revision3 as a disorganized video storehouse, and start talking a bit about them as a company. That word “company” didn’t always feel as strong. if anyone remembers Revision3 from the beginning, they’ll remember thebroken, Systm, and all the other little shows by Kevin Rose and his friends. As G4 slowly killed off TechTV, this is where fans came to get new videos from their favorite personalities without needing an above-basic cable package.
At some point between 2006 and 2008, I noticed a shift in what I was watching. It didn’t feel like as much as a network by and for geeks, but a network by the advertisers for the consumers. I’m not saying that advertising is a bad thing. We all need to pay the bills, but at what cost is advertising okay? I watch a video from Revision3 now, and it feels like more of an advertisement with some show around it than the other way around. Revision3 relaunched with new shows in 2008, and tried going all out to make their mark. You could say they did for a bit but now it just feels tired. Shows come and go in months rather than years, the site is poorly maintained, and I can barely get through an episode anymore. Some of the more popular shows such as Systm have been shutdown. Shows like Tekzilla have been made to rival other shows (Hak5 in this case, which Revision3 later distributed. It is now a shell of itself).Other shows come and go before anyone realizes they’ve been there.
Don’t take my word for how the place is. Read Dave Randolph’s post (later removed from his site). Read David Calkin’s post. Read Wess Tobler’s post.
Read them all.
And now the company is being bought by Discovery. What does that mean?
Another chapter in Revision3 is starting, and I hope it doesn’t end up burning the first part of the book.
Wednesday, May 23rd, 2012
Fourteen days ago, I started the SaveRev3 project (read why here). If anything, it has been off to a good start. Tons of shows are up in one capacity or another, and it doesn’t seem like it will be long before the entire selection of archived shows is online.
Having said that, let’s talk a bit about Revision3. I’d like to describe their setup without using the word “clusterfuck” but that would be pretty difficult. If you are unfamiliar with it, Revision3 keeps a list of “Archived Shows” which houses shows that they don’t produce anymore. This includes shows that went on to do their own thing after Revsion3 such as Epic Fu, notMTV, and The Game Show.
If you browse through this list, you might wonder why it feels like some shows are missing outright, and that is because they are. Plain and sweet, Revision3 for one reason or another has decided to take some shows down, and remove all mentions of them… or so they think. So we know there are shows missing, but how can we get at them? Take any of these missing shows: thebroken, SubSystm, Mysteries of Science Explained, Geekdrome, etc. and do a little googling. You can usually find a dated forum post for the most recent episode, even if it was years ago. From here, use the Wayback Machine for Revision3’s site and migrate to the show’s page (when they had it).
If the Wayback Machine was nice enough to crawl that page, you can get a show synopsis, episode descriptions, download links, and an rss feed. Now, if you click the download link, you will get a 404. Most people probably stop the game here. However, look at any old episode URL and compare it to a new one.
Notice anything? Revision3 has simply changed the url format out over the years (The files are actually just hosted on Bitgravity). Just plug in their current one and you can get at the files.
These episodes have been sitting on the servers, unlinked, for years. They probably just want you to believe the episodes are gone, but for whatever wonderful reason they didn’t actually remove the files.
Using this method, we now have complete runs for every show removed from Revision3.
Let’s talk a bit about shows still on the site. The organization doesn’t get much worse. If you look at any early episodes for a season, the formats for them are a toss up. The Revision3 Gazette doesn’t even have download links for the first two episodes, forcing you to capture ancient flv files from Youtube. A lot of other shows will have the first episode as a small h264, and then quality will catch up on the following episodes. What happened to the first episode? I don’t think it was shot at low resolution. Then there are shows like Indigital where the first episode has a large XviD, but only a small h264. Did nobody want to bother encoding to another format?
The torment continues. One episode of iFanboy Mini returns a 404 for the large h264, even though the episode is linked, it just doesn’t seem to exist. Did someone delete it or name it incorrectly? Did it even exist to begin with? Other shows like Infected are just odd. First few episodes are MP3 only, then they get up to small h264, then back down to MP3 only for a couple of episodes, then small h264 again.
Foodmob Bites had the numbering scheme messed up four years ago, and nobody fixed it. The episode numbers jump up when they feel like it, and the rss feed has been a mess for just as long. Lil’ Internet Superstar doesn’t appear to have an episode 3. Some shows like popSiren Bite are missing episodes in the listing, and they can only be found by searching the site based on description contents, if you can find that.
Up until recently, downloading episodes was a complete pain. You could snag the last 30 or so episodes from the show’s RSS feed, but besides that, you’d have to download them one at a time. There aren’t many series torrents out there, and although some people made some bulk download bash scripts, they’re few and far between.
I enlisted some help, and what I got in return was a fantastic one-line command that will wget all the episodes for whatever show in whatever format almost effortlessly. Now, it does have some problems: every once in awhile a series of episodes won’t download, and it doesn’t take into account that some shows have their early episodes in a lower quality format, but filling those holes is much easier than downloading one episode at a time by hand.
Something else to point out though, is that the Bitgravity cdn will throttle you. On one of my servers, downloads start out at 10M/s, and later drop to 500K/s before building up and leveling around 1.5M/s and then jumping around randomly.
I’d like to give some shout outs to those who have helped out so far in this brief period of time, in chronological order. Moonlit has made sure I keep some sense of my sanity and has done some information gathering on Revision3 Beta. Shinmaryuu has done a mention or two over Twitter. Corrosion has helped put up XviD files of thebroken. RichardR has been downloading/uploading multiple shows. Pat has donated his internet connection. Digip has offered to mail DVDs (though I’ve reckoned downloading would just be faster).
As of right now, I am downloading and uploading from three different connections. I plan on maintaining a dedicated computer with a 2TB external keeping watch over all the files locally and pulling down more.
As always, feel free to join up and spread the word. If you don’t know how to help, come chat with us and we’ll get you on your way.
Tuesday, May 8th, 2012
I have been putting off posting this entry and restarting this project for months now, but it seems like a good time to pull it back to the surface. Five days ago, Discovery announced they were entering an agreement to acquire Revision3. The agreement is to be finalized June 1st, 24 days from now. As expected, they say nothing will be changing, but the true meaning of that isn’t exactly known right now. Who knows who will be making calls about what shows can stay around, and when ones have to stop production. Will someone be making decisions about reducing storage space or bandwidth?
I am publicly starting the Save Rev3 project to capture as much of their content as possible. Not just “in case” something happens, not for “when” something happens, but because someone should have done it a long time ago. Follow all the project updates at anarchivism.org and JOIN UP IF AT ALL POSSIBLE. There is only so much disk space, bandwidth, and energy a single person can have. Even another one or two people can make things go by faster. If you don’t know what to do, come chat with us and we will help you get going.
Let’s get one thing straight right off the bat. I have a long and tiresome relationship with the entity that is Revision3. While I could go on and on about personal politics and business tactics, I’ll try to throw everything into a nutshell for the sake of being tidy. When Revision3 first came into being years and years ago, I was a huge fan. I loved thebroken, I loved Systm, and I loved the idea that there could be a whole television network online that I could turn to when I had nothing else to keep me occupied. Let us examine this a little more. Online television network. Back in 2003 or so, this was a totally new idea. You didn’t have to turn to the corporate-controlled 80 channels you might get on standard cable, you could go find people making content that rested nicely in your collage of interests. Sure, we loved TechTV, but this was new territory. This was, for lack of a better metaphor, the wild west of television distribution. Like most things, Revision 3 started small with a few shows. Few enough that it actually had a time when it fizzled out through 2005 and most of 2006 before relaunching the site later in the year. With the relaunch I had a lot of optimism, but as the years went on I noticed a shift from the hobbyist spirited community to a pseudo-niche corporate-minded organization.
That said, I noticed something else. Little was done to preserve the shows that came and went as the years went by. Some of the earliest shows are completely erased from the Revision3 archives, while others are missing episodes.
Sometime last year, I started saving these shows. I got a a decent amount of files in XviD, but could only grab so many before exhausting my storage space. Later on last year, Revision3 changed the game up by announcing that they were discontinuing certain formats, once again changing my archiving habits. I want to go with high quality files, but not the types that are going to disappear halfway through a show’s run in favor of a competing format
So here is day 0. I am restarting my archiving effort, and looking for all those who want to join me on this long, strange trip. Why archive it? No bullshit: Revision3 does not do a good enough job themselves. They may think they do, but if you’ve browsed as many pages on their site as I have, you’d see just how crazy everything is set up over there. Why do I care? These videos are not just simple internet videos, but building blocks of a whole media revolution that may otherwise be lost. Sure compact discs are just dandy, but I for one want to know about the worlds of cassette tape, and 8-tracks, and vinyl records, and beeswax cylinders before I fully understand where we are today. Some people may just be able to dismiss the past up until the present and take everything at face value, but I’ve never been one to.
Help save a piece of internet history, one episode at a time.
Spread the word.