The Best of 2015

As a nod to @fogus and his blog, Send More Paramedics, I’ve opted to start the annual tradition of recapping the year with the best things I’ve found, learned, read, etc.

These things are listed in no particular order, and may not necessarily be new.

Favorite Blog Posts Read

Not a lot here that I can recall, but this handful stood out as good reads. Some of them I plan to refer back to in the future.

Articles I’ve Written for Other Publications

I’ve tried something different this past year and have worked to write more for others than for just myself. This has been really fun, but has reduced the total number of entries I have written this year in general. I hope to find some more outlets to contribute to with like-minded interests. I like working with small teams like this instead of bouncing ideas around with only myself.

  • Finding Forgotten Footage – An article I did for Lunchmeat Midnight Snack #4 (a print zine) about finding strange VHS tapes with home-recorded footage.
  • Automating Site Backups with Amazon S3 and PHP – An article I did for the now-defunct TechOats website (still sad about that one). As the title describes, I automated backups of my websites using Amazon S3 and a simple PHP script.
  • The New Wild West – An article for NODE about how the internet of things and the sort of always-connected culture opens things up again for a wide variety of attacks. I draw parallels to the 1980’s boom of hacker culture where a lot of stuff was just left wide open.
  • How to Run your Own Independent DNS with Custom TLDs – A tutorial I did for NODE after remembering the failure of the .p2p project and the success of OpenNIC.

Favorite Technical Books Read

I’ve been trying to read a lot more this year to cut through my growing pile of books. I’ve mainly focused on technical books, including books I’ve only been made aware of in 2015 as well as ones that have been on my shelf for years.

  • Garage Virtual Reality – An antiquated virtual reality book from the ’90s touches on a lot of interesting technology from the time, including homemade projects and technological dead ends. The perfect amount of technical instruction and cyberpunk ideas.
  • Hacking the Xbox: An Introduction to Reverse Engineering – An amazing book on reverse engineering. I picked this up around a decade ago, and it was completely over my head. At the time I dismissed it because it was already outdated with the popularity of “softmods” for the Xbox, but picking it up again it is really just a good general book on getting into reverse engineering and the focus on the Xbox is a fun nostalgic little bonus.
  • Cybernetics – A dated and likely obscure text, this book deals with the early ideas of cybernetics and expands into theory on artificial intelligence and neural networks.

Favorite Non-Technical Books Read

  • Microserfs – A fun book that follows a group of ’90s Microsoft employees as they start their own company.
  • Crypto – An incredible look into the world of cryptography, following all of the pioneers and the cypherpunk movement.
  • Dealers of Lightning: Xerox PARC and the Dawn of the Computer Age – My favorite book of the year, a wonderfully- detailed look into the rise and fall of Xerox PARC and all of the completely fascinating things they invented.
  • The World Atlas of Coffee: From Beans to Brewing – I love coffee and this book lets you learn about all the varieties, proper brewing techniques, etc.
  • Ready Player One – A fun dystopic sci-fi book about a civilization obsesses with a treasure hunt and ’80s culture.


Number of Books Read


Favorite Musicians Discovered

  • King Tuff
  • Elle King
  • FFS – Franz Ferdinand and Sparks
  • Devo – Everyone knows “Whip It,” but I’ve been focusing on their first few albums.

Favorite Television Shows

Mr. Robot (2015), The X-Files (1993)

Programming Languages Used for Work/Personal

C, C++, Java, JavaScript, Objective-C, Python.

Programming Languages I Want To Use Next Year

  • Common Lisp – A “generalized” Lisp dialect.
  • Clojure – A Lisp dialect that runs on the Java Virtual Machine
  • Go – Really interested to see how this scales with concurrent network programming.

Still Need to Read

Computer Lib, Literary Machines, Design Patterns, 10 PRINT CHR$(205.5+RND(1)); : GOTO 10

Life Events of 2015

I became engaged to be married.

Life Changing Technologies Discovered

  • Amazon Dash Button – I hacked a $5 button to email me when I press it.
  • Ethereum – An interesting decentralized software platform. Still not entirely sure what to make of it.
  • Microsoft Hololens – I want one after seeing this video. I’ve already supported Oculus for VR, but this is winning me over for AR.

Favorite Subreddits

/r/homelab, /r/retrobattlestations, /r/cyberpunk, /r/homeautomation.

Plans for 2016

  • Get married.
  • Write more for NODE (if possible!), Lunchmeat, or other publicans I find out about.
  • Write an article for 2600.
  • Find my missing Leatherman.
  • Release a mobile app.
  • Do some FPGA projects to get more in-depth with hardware.
  • Continue to flesh out Anarchivism with videos/print.
  • Organization, organization, organization!


See you in 2016!


The New Wild West

This article was originally written for and published at N-O-D-E on August 3rd, 2015. It has been posted here for safe keeping.


A few years ago, I was fortunate enough to work professionally with low energy RF devices under a fairly large corporation. We concerned ourselves with wireless mesh networking and were responsible for tying together smart devices, like light bulbs or door locks installed in your home, into an information-driven digital conglomerate. You know those commercials you see on TV where the father remotely unlocks the door for his child or the businesswoman checks to make sure she left the patio light on? That was us. At the touch of a button on your tablet, miles away, you can open the garage door or flip on the air conditioner. These are products that are designed to make life easier.

In research and development, we view things differently than the stressed-out, on-the-go homeowner might. We don’t necessarily think about what the user might want to buy, but ask the question, “when we roll these things out, how will people try to exploit and break them?” In the confines of a tall, mirror-glass office building, my packet sniffer lights up like a Christmas tree. Devices communicate in short bursts through the airwaves, chirping to one another for all to hear. Anyone with the curiosity and some inexpensive hardware can pick up this kind of traffic. Anyone can see what is traveling over the air. Anyone can intervene.



Things weren’t so different a few decades ago. Back in the ‘70s we saw the rise of the phone phreak. Explorers of the telephone system, these pioneers figured out how to expertly maneuver through the lines, routing their own calls and inching further into the realm of technological discovery. We saw innovators like John Draper and even Steve Wozniak & Steve Jobs peeking into the phone system to see how it ticks and what secrets they could unlock. It wasn’t long before people started connecting their personal microcomputers to the phone line, lovingly pre-installed in their houses for voice communication, and explored computerized telephone switches, VAXen, and other obscure machines — not to mention systems controlled by third parties outside the grasp of good old Ma Bell.

This was the wild west, flooded by console cowboys out to make names for themselves. The systems out there were profoundly unprotected. And why not? Only people who knew about these machines were supposed to be accessing them, no use wasting time to think about keeping things secure. Many machines were simply out there for the taking, with nobody even contemplating how bored teenagers or hobbyist engineers might stumble across them and randomly throw commands over the wire. If you had a computer, a modem, and some time on your hands, you could track down and access these mysterious systems. Entire communities were built around sharing information to get into computers that weren’t your own, and more of these unsecured systems popped up every week. It seemed like the possibilities were endless for the types of machines you would be able to connect to and explore.

Today, many will argue that we focus much more on security. We know that there are those who are going to probe our systems and see what’s open, so we put up countermeasures: concrete walls that we think and hope can keep these minds out. But what about newer technologies? How do we handle the cutting edge? The Internet of Things is still a relatively new concept to most people — an infant in the long-running area of computing. We have hundreds if not thousands of networked devices that we blindly incorporate into our own technological ecosystems. We keep these devices in our homes and on our loved ones. There are bound to be vulnerabilities, insecurities, cracks in the armor.


Maybe you don’t like the idea of outlets that know what is plugged into them or refrigerators that know when they’re out of food. Maybe you’re a technological hold-out, a neo-luddite, a cautious person who needs to observe and understand before trusting absolutely. This may feel like the ultimate exercise of security and self-preservation, but how much is happening outside of your control?

When the concept of ubiquitous computing was first developed by Mark Weiser at Xerox PARC in the late ‘80s, few knew just how prominent these concepts would be in 25 years. Ubiquitous computing pioneered the general idea of “computing everywhere” through the possibility of small networked devices distributed through day-to-day life. If you have a cellular telephone, GPS, smart watch, or RFID-tagged badge to get into the office, you’re living in a world where ubiquitous computing thrives.

We’ve seen a shift from the centralized systems like mainframes and minicomputers to these smaller decentralized personal devices. We now have machines, traditional personal computers and smart-phones included, that can act independent of a centralized monolithic engine. These devices are only getting smaller, more inexpensive, and more available to the public. We see hobby applications for moisture sensors and home automation systems using off-the-shelf hardware like Arduinos and Raspberry Pis. The technology we play with is becoming more independant and increasingly able when it comes to autonomous communication. Little intervention is needed from an operator, if any is needed at all.

For all of the benefits we see from ubiquitous computing, there are negatives. While having a lot of information at our fingertips and an intuitive process to carry out tasks is inviting, the intrusive nature of the technology can leave many slow to adopt. As technology becomes more ubiquitous, it may also become more pervasive. We like the idea of a smart card to get us on the metro, but don’t take so kindly to knowing we are tracked and filed with every swipe. Our habits have become public record. In the current landscape of the “open data” movement, everything from our cell phone usage to parking ticket history can become one entry in a pool of data that anyone can access. We are monitored whether we realize it or not.


We have entered uncharted territory. As more devices make their way to market, the more possibilities there are for people to explore and exploit them. Sure, some vendors take security into consideration, but nobody ever thinks their system is vulnerable until it is broken. Consider common attacks we see today and how they might ultimately evolve to infect other platforms. How interesting would it be if we saw a DDoS attack that originated from malware found on smart dishwashers? We have these devices that we never consider to be a potential threat to us, but they are just as vulnerable as any other entity on the web.

Consider the hobbyists out there working on drones, or even military applications. Can you imagine a drone flying around, delivering malware to other drones? Maybe the future of botnets is an actual network of infected flying robots. It is likely only a matter of time before we have a portfolio of exploits which can hijack these machines and overthrow control.

Many attacks taken on computer systems in the present day can trace their roots back over decades. We see a lot of the same concepts growing and evolving, changing with the times to be more efficient antagonists. We could eventually see throwbacks to the days of more destructive viruses appear on our modern devices. Instead of popping “arf arf, gotcha!” on the screen and erasing your hard drive, what if we witnessed a Stuxnet-esque exploit that penetrates your washing machine and shrinks your clothes by turning the water temperature up?

I summon images from the first volume of the dystopian Transmetropolitan. Our protagonist Spider Jerusalem returns to his apartment only to find that his household appliance is on drugs. What does this say about our own future? Consider Amazon’s Echo or even Apple’s Siri. Is it only a matter of time before we see modifications and hacks that can cause these machine to feel? Will our computers hallucinate and spout junk? Maybe my coffee maker will only brew half a pot before it decides to no longer be subservient in my morning ritual. This could be a far-off concept, but as we incorporate more smart devices into our lives, we may one day find ourselves incorporated into theirs.


Just as we saw 30 years ago, there is now an explosion of new devices ready to be accessed and analyzed by a ragtag generation of tinkerers and experimenters. If you know where to look, there is fruit ripe for the picking. We’ve come around again to a point where the cowboys make their names, walls are broken down, and information is shared openly between those who are willing to find it. I don’t know what the future holds for us as our lives become more intertwined with technology, but I can only expect that people will continue to innovate and explore the systems that compose the world around them.

And with any hope, they’ll leave my coffee maker alone.



Philosophy of Data Organization

I would be a liar if I said I was an overly organized person. I believe that like things should be grouped together and everything is to have its place, but I follow something of a level of acceptable chaos. Nothing is organized completely, and I don’t really believe it is possible to have complete organization on a large enough scale. Complete organization is likely to cause insanity.

When I first started accumulating data, I quickly outgrew my laptop’s 80 gigabyte hard drive. From there I went to a 150GB drive, then a pair of 320GB drives, then a pair of 1TB drives, then a pair of 2TB drives, and from there I keep amassing even more 2TB drives. As I get new drives, I like to rotate the data off of the older ones and on to the newer ones. These old drives become work horses for torrents and rendering off video while new drives are used for duplicating and storing data that I really want to keep around for a long long time. The system is ad-hoc without any calculated sense of foresight. If I had the money and planning, I’d build a giant NAS for my needs. For now, whenever I need more space, I just buy another pair of drives and fill them up before repeating the cycle. This doesn’t scale very well and I ultimately around 25TB of storage scattered across various drives.

A few months ago, I was fortunate enough to take a class on the philosophy of mind and knowledge organization. A mouthful of a topic, I know, but it is more simple than it seems. The class revolved around one main concept: classification. We started with concepts put forth by Greek philosophers on how to organize knowledge via the study of knowledge: epistemology. We start out with concepts put forth by Socrates, Plato and Aristotle. Notably, university subjects were broken into the trivium (grammar, logic, and rhetoric) and later expanded with the quadrivium (arithmetic, geometry, music, and astronomy) as outlined by Plato. These subjects categorized the liberal arts, based on thinking, as opposed to the practical arts, based on doing. These classifications were standard in educational systems for some time.

The Trivium

A representation of the Trivium

Aristotle reclassifies knowledge later by breaking everything into three categories: theoretical, practical, and productive. This is again broken down further. Aristotle breaks “theoretical” into metaphysics, mathematics, and physics. “Productive” is broken into crafts and fine arts. “Practical” is broken down into ethics, economics, and politics. From here, we have a more modern approach to knowledge organization. We see distinctive lines between subjects which are further branched into more specific subjects. We also see a logical progression from theoretical to practical, and finally to productive to ultimately create a product.

An outline of Aristotle's classification

An outline of Aristotle’s classification

More modern classifications pull directly from these Greek outlines. We can observe works by Hugo St. Victor and St. Bonaventure which mash various aspects of these classifications together to create hybrid classifications which may or may not be more successful in breaking down aspects of the world.

An interpretation of St. Bonaventure's organization

An interpretation of St. Bonaventure’s organization

What does this have to do with data? Data, much like knowledge, can be organized using the same principles we have observed here. Remember, the key theme here is classification. We are not simply concerned with how to break up knowledge, but anything and everything that can be classified.

Think of all the possible ways you could organize films or musical artists, or even genres of music. It can be a daunting thing to even imagine. As an overarching project throughout the course, we developed classifications of our own choice. I chose to focus on videotape formats, and quickly created my own classification based on physical properties. I broke down tapes into open/closed reel, tape widths, and format families. While it might not be the best classification, I tried to approach the problem in a way that was open to using empirical truth (conformity through observations) in a way which would allow a newcomer to quickly traverse the classification branches to discover what format he is holding in his hands.

An early version of my videotape classification

An early version of my videotape classification

Classifications like this are not uncommon. Apart from the classifications of knowledge put forward here already, classifications have been used by Diderot and d’Alembert to create the first encyclopaedia in 1759. This Encyclopédie uses a custom classification of knowledge as its table of contents. While generalized to an extent (it does fit one page), it could be expanded upson infinitely.

Encyclopédie contents

Encyclopédie contents

A contemporary way to organize knowledge arrives in a familiar area: the Dewey Decimal System. Though Dewey’s system has been adopted globally as the de facto method for organizing print media, can can we apply this same system to our growing “library” of data? The short answer is no, not without some modification, though modifications have plagued Dewey’s system since its inception.

To understand how we can best organize our data, we must first understand the general concepts of the Dewey Decimal System. Within the system, different categories are defined by different numbers. 100 may be reserved for philosophy and psychology while 300 may be used for social sciences, and 800 for literature. the numbering system here is intentional. Lower numbers are thought to be the most important subjects while higher numbers are less important. These numbers are broken down further. 100 might be broken into 110 for metaphysics, 120 for epistemology, etc. with each of these being broken down again for more finite subjects.

This is just another classification, but it has its faults. The size of a section is finite as the system is broken up into 10 classes which are then again broken down into 10 divisions, and finally 10 sections (hence decimal). However, we never really accounted for the growth of new and expanding topics. As subjects emerge like computer science, which Dewey never could have imagined, we throw works like these into unused spaces. Computer science in particular is infamous as it now occupies location 000, which in the system would make it seem more important than any other subject in the entire system. Additionally, we see a loss in physical ties to the system as libraries are intended to to organized along with the system: lower numbers on the first floor, higher numbers on the higher floors. Dewey’s system is constantly being modified as new works emerge and finding any consistency between different libraries could be controlled by one librarian who chooses whether or not to implement a change at any given time.

A simplified example of Dewey's system

A simplified example of Dewey’s system

While a modified version of Dewey’s system might make sense for data (as well as being somewhat familiar), we have to consider another problem which plagues the classification: titles that can occupy more than one section. Suppose that I have a book about WWII music. Do I put this book in music? Does it go in history? What other sections could it fall into? We have few provisions for this.

Data is no different in this sense. Whether I have a digital copy of a book as would be found in Dewey’s system or a podcast or anything else, there is always the potential for multiple areas a work can fall into. If you visit the “wrong” section where you might expect an object to be, you don’t have any indication that it would be somewhere else just as suitable.

What are we to do in this case? While I like to break my data down into type of media (video, audio, print, etc), I find the lower levels to get more fuzzy. Let us consider a subject which I am revisiting in my own projects: hacker/cyberpunk magazines. Even if we only focus on print magazines, we still have problems. We can see the concept of “hacking” coming from more traditional clever programming origins (such as in Dr. Dobb’s Journal), or evolved from phreaker culture (such as in TEL), or maybe from general yippie counterculture (such as in YIPL). Additionally, we can see that some of these magazines feature a large number of overlapping collaborators which make them feel somewhat similar. We also may observe that magazines produced in places like San Francisco or Austin also have a similar feel but might be much closer to other works that have no physical or personnel ties. Further, what about publications that started as print and then went over to online releases? More and more possible subgroups emerge.

At this point, we might consider work put forth by Wittgenstein which is based off of the “family resemblance theory.” The basic idea behind this theory is that while many members of a family might have features that make them resemble the family, not one feature is shown in all the members who have family resemblance. Expanded, we can say that while we all know what something means, it can’t always be clearly defined and its boundaries cannot always be sharply drawn. Rosch, a psychology professor, took Wittgenstein’s concept further and hypothesized that “the task of categorization systems is to provide maximum information with the least cognitive effort.” She believes that basic-level objects should have “as many properties as possible predictable from knowing any one property.” This means that if something is part of a category, you could easily know much more about it (if you know that 2600 is a hacking magazine, you’ll know there are likely articles in it about computers). However, superordinate categories (like furniture or vehicle) wouldn’t share many attributes with each other. Rosch concluded that most categories do not have clear-cut boundaries and are difficult to classify. This goes on to show the concept that “messiness begins within.” We get a contrast from Aristotelian “orderliness” because messiness shows that we can’t put things in their place because those places are just where things “sort-of” belong. Everything belongs in more than one place, even if it is just a little bit. We see that order can be restrictive.

This raises the importance of metadata: data about data. While my media might be organized in such a classification that doesn’t allow for “double dipping” (going against concepts by Rosch), we can utilize the different properties that pertain to each individual object. Consider many popular torrent sites which utilize crowd-sourced tagging systems. Members can add tags to individual pieces of media (which can then me voted on as a way to weed out improper tags) which allow the media to show up in searches for each tag. We see a similar phenomenon in websites such as Youtube which allow tagging of videos for content, though not in a crowd-sourced sense or the Internet Archive which supports general subject tags as well as more specific metadata fields.

Using this metadata method and my previous example, it’s easy to find magazines by location, authors, subject, contents, age, and a long list of other attributes. We can apply this to objects that aren’t the same format; there are examples of video, audio, and print that pertain to the same subjects, authors, etc. This isn’t an impossible implementation. Considering further the Internet Archive, we see thousands upon thousands of metadata-rich items which are easily searchable and identifiable. However, the Internet Archive also suffers from a lackluster interface. It might be easy to find issues of Byte magazine, but it is a lot more difficult to figure out what issues we are missing or see an organizational flow more akin to a wiki system (though both systems lend themselves well to items being in more than one place). A hybridized system like this would be an option worth exploring, but I haven’t seen an ideal execution of it yet.

While this concept of a metadata-based organizational system isn’t a fool-proof solution, it can certainly be seen as a step in the right direction. We must also consider the credibility of those who decide to make contributions to metadata, especially on a large-scale public system. Consider the chaos and political makeup of how Wikipedia governs editing and then you’ll start to get an idea. While I’d like to implement a tagging system for my own personal media library (with my own tagging at first and the possibility of expansion), I am limited by my current conglomeration of hard drives scattered to different parts of the house, usually powered off. My next storage solution will take these ideas into planning and execution, making my data much easier to traverse. I will however have limitations as I won’t have many people perpetually reviewing and tagging my data with relevant information.

That said, the idea of being able to make my data more accessible is an exciting one, and increases portability of the data as a whole if I ever need to pass it on to others. As my tastes evolve and grow, so will the collection of data I hold.

With any hope, my organized chaos will ultimately become a little more organized and a little less chaotic.

With any luck, you’ll be able to browse it one day.


Ghost in the Machine: Your Digital Afterlife

This article was originally written for and published at The New Tech on July 9th, 2013. It has been posted here for safe keeping.


On January 11th, Aaron Swartz passed away. If you’re not familiar with who he was and what he did, take a minute right now and look him up. A lot of focus was put on the circumstances of his death along with what he accomplished in life, and this seems to overshadow something that stood out to me: how to handle his legacy. Specifically, how he wanted his legacy to be handled.

Swartz created a simple web page in 2002 about how to handle things if he were to be “hit by a truck.” Who would take over his website? Where would his source code end up? He created an electronic will. The idea of a will is nothing new. Most people create a document outlining how their assets will be divided up when the time comes- it just makes things operate more smoothly. But what about in the electronic world? Surely we mark who will get our house but what about who gets our website? It sounds amusing to think about or even entertain the idea. We allocate or physical property, things that can be defined in dollars and cents, but hardly consider our intellectual property.

If you haven’t had a Facebook friend pass away yet, you’re likely in the minority. It’s sad, of course it’s sad, but it needs to be talked about. If you have ever had a Facebook friend pass, you may observe a cycle where his/her profile is used first as a memorial and then eventually deactivated all-together. These are my experiences. I understand and empathize with the feelings of the family in these circumstances, but to me this seems a little like burning all of a loved one’s possessions with the ease of a single mouse click.

These are the two ends of the spectrum.

It’s important to let go and move on, but it’s also important to remember and honor. In a basic sense, I apply the same fundamental ideas towards the death of a person as I do towards that of a technology. Most are quick to push the old out of thought, but the few make a move to preserve. I preserve. It’s just my nature.

Swartz’s situation resonated with my own beliefs. If I were to be hit by a truck tomorrow, what would happen with my stuff? My digital stuff. I run a fair number of websites, I rent a VPS and a dedicated server, and I have bills, Amazon S3, service subscriptions. If I go, they eventually do too.

I’d like to tell you that I have a contingency plan, but I don’t. I haven’t reflected fully on the logistics of it. Could I think of people to take over my digital stuff after I’m gone? Of course, but would they want to? When someone dies and it becomes your responsibility to handle their belongings, it’s not typically a drawn out process. You keep some stuff, you toss some stuff, but you don’t normally end up with something that needs to be maintained and worked through. Websites take a fair amount of time and money. Storage, while getting cheaper, is still expensive for the hobbyist. There’s unavoidable maintenance.

That said, I would hope that my online persona remains long after I do. Forum accounts, Facebook information, Twitter posts, etc. should survive as long as possible. I want everything to be available to anyone who needs it. Hand over my source code and pick apart my log files.

Open source my life.

If I’m not going to work on it anymore, I’d like to give that ability to anybody who is interested.

To have these things removed, stripped from the world, is just nonsensical to me. Someone’s interesting and original work disappearing because nobody knows how to or doesn’t want to handle handle it? Nothing upsets me more than something like that. It’s akin to tearing pages out of every history book. In our modern world, people are quick to think that things last forever. Digital artifacts can go missing overnight.

We shouldn’t be worried so much anymore about having something embarrassing stored online forever, we should be more worried about something important disappearing tomorrow.


What Administrating a BitTorrent Site Taught Me About Project Management

This article was originally written for and published at Medium on May 18th, 2013. It has been posted here for safe keeping.

In my sophomore year of college, I became an administrator of a BitTorrent website. It’s not nearly as shady as it sounds. In fact, it was a small and completely legal operation. Three administrators, one server, and hard drive after hard drive full of Creative Commons-licensed content.

Now, I’m lucky enough to attend an undergraduate school with a strong internship tie-in. We spend half of the year slaving away on our school work while the other half is spent in one of those real-world jobs. Lather, rinse, repeat for three years. On the academic side, we take our specialized engineering classes, our project management classes, our technical communication classes, our how-to-work-with people classes.

I didn’t take many of those yet. They first year and change at the university is mostly populated with weed-out classes and introductory curriculum akin to a secondary school elective class or two. At this point in my life,I didn’t know what makes a good project and I didn’t know what makes a project good. I didn’t know how to communicate effectively or work as part of a group. I didn’t know about Gantt charts, or deliverables, or development practices.

As I mentioned, there were three of us. One administrator I had met via online chat some months prior in a public channel. He was a decent guy, and the linkage between myself and the mysterious third administrator who I had never spoken with but was providing us with a server. We all came together, communicating with each other in a strictly online format. Geographically separated, what did it matter with email and a few common hours when we all happened to be awake at the same time? We didn’t have structure or a real thought-out plan. No documents or task lists or meetings to touch base. We carved out and constructed bits and pieces when we felt like it and waited for each other to catch up before charging forward again full steam.

It happened to be winter break, and I had plenty of free time to devote. After we eventually got the site up and operational, I spent days filling it with uploads and tutorials, configuring and reworking plugins and style sheets, setting up social networking accounts, and more or less doing my damnedest to make it ready for prime time. Then, we got a pay-off. A file-sharing blog picked up on the site and did a piece. Within 48 hours, news spread and we had some 3,000 members. We were being reblogged and discussed in forums. We were growing by the hour.

Sounds great, huh? It wasn’t.

While we had all been united in our quest to launch a fantastic niche torrent site, we quickly split at the seams. While I tried my best to keep a steady flow of content being uploaded to seed the site for new users, the other admins didn’t seem as compelled to. One simply disappeared for weeks at a time while another decided it would be a good time to ask for donations and not do much else. Our chat sessions together got shorter and eventually vanished completely. The site stagnated except for a small group of hopefuls that were uploading and contributing, but it amounted to too little. We fell apart. We were broken.

One day, I made a passing comment to a user about how I’d like to rebuild and relaunch the site, and then found myself stripped of my administrative permissions. I contacted the one administrator I had known prior to starting the project, and he just shrugged off the situation as weird before reinstating me into the ranks. It was too late, though. I ended up deleting myself of my own accord a week later.

I completely removed myself from the project, but that doesn’t mean I left empty-handed. I departed with lessons forged from mistakes and successes. What worked, and what didn’t. I learned the need for defining a project scope and keeping open the lines of communication. I learned the importance of meeting regularly and setting goals and being assertive. I learned sacrifice and when to cut your losses and move on.

Each one of these lessons followed me as I went from internship to internship and class project to class project. Academia can teach you a good amount about how to be a developer, but falls a little short when it comes to how to work with real people in the real world.

To learn that, well—you just need to experience it.


Documentaries I’d Like to See Before I Die (Or Everyone Forgets)

I’d like to think that I have something of a second nature when it comes to whether or not there is a documentary made or in production for any of my disjointed hobbies and interests. It’s not one of those skills you showcase in your job interview, but I seem to have this knack for religiously crawling the web in search for films I think I’d enjoy. Surprisingly, and to my great pleasure, a lot of these fringe interests I posses already have films about them. Awesome. However, there are a few that simply do not- or, have a film that doesn’t satiate my particular appetite.

So, for my sanity, I made a list of the topics I’d personally like to see filmed. And, in some cases, some topics I’d probably find gratification in filming myself.

Written below is that very list. Think of this more as a way of me getting the thoughts from my head to paper as opposed to a list of full-bodied explanations and fleshed-out ideas.

Demoscene. There are already a few demoscene documentaties out there. For example, The Demoscene Documentary is about the demoscene in Finland and Moleman 2 is a demoscene documentary focusing mainly on Hungary. While these are in fact good films, they each have a specific scope. From what I gather, the demoscene can be radically different from country to country, making it difficult to understand as a whole when only presented with a few of its parts. I’d propose an episodic piece showcasing the demoscene in a variety of countries – each country having its own segment. While these existing documentaries have touched on Finland and Hungary, there are still Germany, USA, Denmark, and Norway to consider (and probably others).

Bitcoin & Digital Currency. We’ve all heard of Bitcoin by now, especially as it makes waves at it’s current high value. However, Bitcoin itself has an interesting past and makes an interesting statement. If you do any detective work about how Bitcoin came to be, you will be sucked up into a mysterious story about how nobody knows the identity of the creator or what happened to him. The conspiracy theories are vast and plenty. We also touch on the interesting issue of an unregulated worldwide currency, governments attempting regulation, bitcoin-mining malware botnets, attacks on exchanges, etc. How about how crazy some people go with their mining setups? Dozens of caseless computers fillied with graphics cards- a cyberpunk daydream turned reality. How about using FPGAs and these new ASIC rigs? Now, that’s just bitcoin. There are numerous other digital currencies out there such as the newer litecoin, or even e-gold (Created in 1996). Digital currency has been around longer than most people think.

Cypherpunk. The cypherpunk movement does for cryptograhy what the cyberpunk scene did for personal computing. While cypherpunks have been around for decades, the interest within the scene has been renewed and pushed towards the mainstream more recently. Going back to “A Cypherpunk Manifesto” and the cypherpunk mailing list, we see early discussions of online privacy and censorship, paving the way for Bitcoin, Wikileaks, CryptoParty, Tor, 3D-printing of weaponry, etc.

Usenet. Started in 1980, Usenet is a system for users to read and post messages. Usenet can be seen as the precursor to internet forums, and is much like a Bulletin Board System in theory except it is distributed among many servers instead of a central authority. As time goes on, Usenet continues to grow in bandwidth usage, now generating terabytes of traffic a day. This is mostly through binary file transfers as opposed to messages. Despite many main ISPs deciding to remove Usenet access from their internet services, many still seek out paid access.

Pirate Radio UK. While Pirate Radio USA and Making Waves do a fantastic job at covering pirate Radio in the US, I haven’t seen much of an effort to show off pirate radio in the UK. From what I’ve gathered, there are an uncountable number of pirate radio stations across the pond, and it’s a different game when compared to the US. At the peak of pirate radio’s popularity, there were near 600 stations active in the UK while there are presently 150, mostly based in London. Here’s a mini piece from Vice.

Darknet. Not in regards to file sharing. More covering the darknet as a blanket term for an independant or ad-hoc network with some sort of disconnection from the internet. Considering topics like Hyperboria and CJDNS, Tor and the Deep Web, Meshneting for fun or necesity, Tin-Can, and so-on. As the hardware becomes less expensive and more devices have networking abilities, creating a scalable network becomes a more achievable task.

Dyson. I feel that James Dyson doesn’t get as much credit as a revolutionary engineer as he deserves. Dyson focuses on improvement: taking the wheel and making it better. No pun intended, but his first success was the creation of a fiberglass wheelbarrow that used a ball instead of a wheel. Afterwards, he famously created over 1000 prototypes for a new vaccuum cleaner using cyclone technology after noticing problems with his Hoover. Dyson repeatedly uses creative thinking and pulls inspiration from unlikely sources.

Raspberry Pi. While the Raspberry Pi was not necessarilly a unique and new concept, it was certainly one of the most well executed. We have seen other incarnations of plug computers such as the Beagleboard or the Sheevaplug, but the Raspberry Pi’s addition of integrated video sets it apart. And, at the price of $30, makes it incredibly affordable. Many would argue that what makes the Pi so special is the community that has formed around it, and not necessarily the hardware that ties it together. Everyone stretches their imagination and expertise: if it can be on the Pi, it should. Aside from the community, the Raspberry Pi Foundation has been done an incredible job at cultivating the technology and inspiring the next generation of young programmers and hardware hackers.

Kickstarter. There have been documentaries in the works that focus on crowdfunding, but I’m not as interested in the crowdfunding movement as much as I am in Kickstarter the company. While Indie GoGo has been around for a longer time, they do not seem to be held together as tightly. Kickstarter seems like not only an interesting company, but one that holds itself, and those who utilize its services, to a high standard.

QUBE. Here’s an odd one for that likely nobody has heard of. QUBE was the first interactive TV station, started in 1977 in Columbus, Ohio. Residents who subscribed to the cable service received a device that looked something like a calculator that allowed them to communicate back to the station during shows. Aside from the interactive feature, QUBE was on the forefront of pay-per-view programming and special interest content. QUBE soon went bankrupt and dissolved in the early 1980s. As a bit of an aside, I think I actually tried contacting the webmaster of that site a while back to ask if I could get a copy of the “QUBE DVD” for archiving but didn’t get a reply. Let’s hope he/she runs Webalizer or Google Analytics and sees some referrer traffic. Maybe it’ll be enough to spark a conversation.

So here ends my list. While the majority of these ideas are feasible, I can’t help but think a few might end up slipping too far and too fast into obscurity before their time. Other ideas on here might be too early in their lives. Doing something now, or even within the next decade, would only show a small part of the eventual picture.

Do I expect any of these to be made? Not particularly. But you never know.

Everyone gets lucky once in a while.


Other People’s Stuff

If you collect as many things as I do, you end up with some stuff you’re not supposed to have. In this case, I’m not referring to stuff that is illegal or stuff that is unreleased. I’m talking about other people’s stuff. Personal stuff.

In a broad sense, the whole used market is a little bit bizarre when you look at it abstractly. As we live in a disposable culture, anything someone might buy has its own story. You don’t know how many hours were put into that dusty SNES with a five dollar sticker on it. How it contributed to sibling rivalries or became an item to bond over with the girl next door. We’ll usually never really know the extent of these stories.

Sometimes we accidentally inherit the stories.

In Philadelphia, there are many little second hand shops that line the grid-like streets. Tucked into corners, away from the tourist traps, these stores don’t feel like your normal thrift shops. These places lean a little more towards collectibles: antique books, crates of records, obscure (but not rare) VHS tapes- you get the idea. Some of these shops also sell photographs, but not of any famous attractions or curios of the city. They’re family photographs. Weird pictures of people posing outside their houses, sitting with their pets, or just acting goofy. Private pictures. Who would have ever thought they’d end up at a store somewhere?

So every one in a while, I buy a few. I have them scattered along the edges of my bedroom mirror. Who are these people? I’ll never know.




This concept doesn’t simply apply to pictures. From one auction, I got a lot of around thirty 7-inch tape reels. While a lot of them were simply recordings of the radio that could be played back for hours, some of them appeared to be homemade recordings. One I remember in particular appeared to be a recording of some sort of part, complete with almost unintelligible voices and faint background music. Something never meant to get out this far. A memory I own that isn’t mine.

Home movies are another area. On occasion, I’ve purchased VHS camcorders with tapes still inside. Rarely though will I find something captivating. Usually, there will be a short video of a newborn baby or the typical “I’m testing out the camera” tape where people pan around their living rooms.

Occasionally though, I’ll find something more interesting.

One flea market I frequent in Delaware usually has a lot of vendors from house clean-outs. They’re easy to spot: Giant rented truck with several dozen cardboard boxes packed full of everything imaginable. No rhyme or reason here: folded up clothes, kitchen appliances, weathered books, etc. Almost as though a family was packed away into boxes to be sold for five dollars a pop. Anyway, while most of these boxes are filled with junk, I’ve found my fair share of interesting objects from them. At one point, I came across a box of hand-labeled VHS tapes. I didn’t know what exactly they were, so I took them home and played them. I found myself with what appears to be recordings of an amateur band (or several) from the mid 1990’s.

So what did I do with them? After setting them to the side for some time, I decided it was best to transfer them and let people see them. Currently, I have one already online with more on the way. Here’s a link to go watch it. Who are these people? What’s the name of the band? I have no idea. The band might be named “Triple X” but nothing seems to enforce that. Maybe I’ll run across something as I keep going, or someone will stumble upon this video and recognize it. There’s a lot more footage to look through.

So this gives you something to think about. Be mindful of those little personal artifacts that you keep around. Those memories frozen in time. Who knows where they will end up one day.

And who knows if some 20-something punk will eventually put them on YouTube.


Beggars Can’t Be Choosers, But They Can Be Social Engineers

Two weeks ago, a beggar approached me while I was washing my hands in the bathroom of a train station. He went on to tell me a story about his troubles and how he needed some money. I of course gave him a few dollars and we both went on with our days. The whole situation was eerie. Going to school in the city, I end up dealing with a broad range of people in any given day. This was different.

If you’re like me, then you spend a considerable portion of your day dissecting. Whether it be conversations you had last month with someone, physically dismantling a piece of hardware, or deconstructing an abstract idea- the method is the same. So as I walked up to my train platform, I couldn’t help but pick apart the scenario bit by bit.

Here’s a little run-through of what happened. I go to wash my hands. As I soap them up, a man comes over and says, “hey man, how you doing?” I return the pleasantry and go about washing my hands. He then starts up a story, just believable enough to hook you in. I get told that he’s here with his daughter and he’s been here a few days trying to get back home but the tickets cost $57. Last night, he was hit in the face with a gun (he shows me the bloody gash on his nose) and he wants to get out of this city. He reaches into his jacket and pulls out a stick of deodorant to show me that he’s trying to stay presentable and, in his words, not lying to me. I tell him I’ll give him a few bucks as I walk over to the hand dryer and hand over a little cash. He asks if I can spare ten dollars, I tell him no can do, and he rushes out of the place before I can even put my wallet back.

All of this happened in about 30 seconds. He made three dollars.

Let’s break it down now. The first thing that seemed a little off was that he showed up when I was the only one in the place. The way the bathroom is set up, he must have came from the stall area. Could he have been waiting until I was the only one left? Why he picked me is a mystery. I wasn’t dressed nicely in any sense of the word, and considering the place fills with business men in suits every few minutes I’m an odd choice. I don’t look like I have much walking around money. Moving on, he initiated the conversation in a friendly matter and did so as I was soaping up my hands. I couldn’t leave right then and there, I had to get the soap off my hands so I was essentially trapped for him to tell his story. The train ticket price is accurate, and we’re in the male restroom  so his daughter doesn’t need to be around. That checks out. He’s dressed in reasonably worn clothes, nothing tattered but nothing too new. He looked a little scruffy, so that would be able to work with his story of being here a few days. The gun story raises a few questions. We’re in a relatively safe part of the city with police everywhere, for about 12 blocks in each direction. So if he was trying to get home, where did he get assaulted? Did he get robbed? If he got robbed, why would he spend money on deodorant? Either way, when a man approaches you out of nowhere in a confined bathroom, says the word “gun,” and then reaches into his jacket you tend to go along with what he says and not call him out on the spot. Next, he said that he wasn’t lying to me. If you ask me, that’s a pretty good sign that he is. His asking for more money is something I deal with on a daily basis from beggars and the homeless, so no surprise there. Lastly, his getaway was swift just as more people came into restroom.

At this point, you’re wondering why I’m bringing this up.

Under normal circumstances, I would just dismiss this as a normal random incident. However, as I reflected on his story I remembered one from two years earlier right outside the same building. Different guy, similar story. This guy, I passed once and he was talking to someone. When I came back around that way an hour or so later, he picked me to talk to next. He told me a story about how his daughter is in the hospital and he’s been stuck in the city for days. He’s just trying to get home so he can get cleaned up. As he started to get emotional, he asked me if I could spare, in his words, four to five dollars. I gave him a few bucks and he thanked me profusely before walking away. He’s in the city, so I will guess his daughter is at the nearby Children’s Hospital. However, why is he outside of this train station? The hospital has a train station two blocks away, while this one is about 10 blocks away. I saw him earlier, so I couldn’t have been the only one to stop for him in the last hour, and local train tickets are only between $4 and $8.

This started to get me thinking about social engineering more than I had in any recent time. Less am I seeing the traditional beggar simply asking for change and more am I getting a well rehearsed story. If we look back, the confidence trick has been used for decades to coax the unsuspecting into giving anything from money to information. Most people have cons aimed at them everyday, but shrug them off as they’ve now become part of everyday life.

But, is there anything we can learn from the creative ways beggars ask for money?

The things these stories had in common are the points to pay attention to. Starting with the approach, the wanting party makes the first move and starts the story out in a friendly way. This will hook in the target because they are wondering why they are being approached, and a guy with a smile can’t be bad right? The story then has an emotional element that makes the target want to help the wanting party. The story itself is well rehearsed, which makes it sound more natural. Speed is also crucial here as the wanting party will say everything he needs to before losing interest from the target. The wanting party will then identify exactly what they want and ask for it from the target specifically. Then after getting something, they make the clean getaway before anyone has a chance to think about what just happened.

We can draw parallels here with traditional stories of social engineering. If you think about any instance of phone phreaks calling up the phone company, you’ll see the same flow. They initiate a conversation politely and bring up a scenario that makes the telco worker want to help them out. They explain the issue swiftly and focus on the information they want. Whether they get what they want or not, they close the conversation quickly and move on.

It’s essentially the same set of fundamentals.

If anything, this whole scenario just got me thinking a little more on psychological manipulation. Hopefully, the next time you’re approached and asked for money, you’ll be thinking a bit too.



DIY Archiving – A Primer

This article was originally written for and published at The New Tech on July 8th, 2012. It has been posted here for safe keeping.

So maybe you want to be an archivist but don’t know where to start. I don’t claim to be an expert, but I have learned a few things going down this path that I can share. Let’s break this up into two main sections: digital media and physical media. No matter what you are archiving, you should first pick out what you’re going to save. You don’t have to put too much consideration into this step. You can be passionate about a subject you wish to preserve, be helping a group of people, or doing it for the hell of it. Archiving is at it’s base both a way to ensure survival and a way to fill a hole left forgotten.

In the digital world, you’re going to be focused on downloading, storing, and uploading. Let’s start with downloading as you’re going to want to get your hands on some media. I started on a Windows computer, downloading directly with a browser. You can get your hands on some stuff simply with a DownThemAll plugin and a lot of free time. For streaming content I turn to the video downloader plugin for Firefox or Replay Media Catcher. This lets you pull videos from all your sites like Vimeo or Youtube. Sometimes things aren’t as easy as clicking a file to download it. You may have to use download sites, ftp servers, Bittorrent, or something less standard. You never know. Get to know how to use jdownloader, an ftp client like Filezilla, and a Bittorrent client like uTorrent. You might find a whole slew of content on some obscure site and you need to have the tools to dig it out. Moving on to more Unix-like systems, learn all you can about wget, curl, grep, and bash scripting. I’m not going to cover how to use all of these tools, but with a little practice there are few things you can’t get when you use them in tandem. You would be surprised how simple it is to whip up automated processes that do everything you want with just a few sophisticated commands. Also, be on the lookout for more specified tools. For example, I found a fantastic tool for downloading Youtube videos called youtubedl. If there is something out there to be downloaded, there is usually a tool for the job.

When you get the data, you’re going to need to hold it. I was originally downloading everything locally, and still like to keep local copies of data I retrieve. Always keep your data on at least two drives, and preferably buy your drives in pairs so you can easily stick with this rule. You can never have too much storage. I currently have 15TB locally just for storing archived media. When it comes to other storage mediums, I’m not easily swayed. Data tape is expensive to adopt, and cloud storage lacks stability. In earlier days, I had only an 80GB drive, so I would back up to DVD+Rs. A lot of people will tell you that your burned media will go bad after about 6-10 years, but I have yet to have a disc become unreadable. I will say that I’ve had a lot of luck with Verbatim discs. I would coaster many discs by other brands when burning, but have only ever had one Verbatim fail on me. Stick with what your budget is, the price of hard drives are only going down, but if you’re a kid on a budget a spindle of DVDs can help in a pinch. Also, keep an eye on solid state drives. While I have yet to adopt them, they are the new thing in mass storage, though the price is still a bit steep.

So now you want to share your data. You have many options to consider. I’ve been using The Internet Archive most recently to place files which should be saved. Depending on your content, this may not be the best option for you. A few of the techniques I mentioned before to download from can be good options for quick data dispersal. For example, setting up a torrent for your files can be done in minutes, and fast FTP/HTTP servers can be rented for however much money you want to spend. The main points here though are longevity and redundancy. You want your files up for a long time, and you want them to stay online somewhere if one server takes a tumble. While torrents alone are terrible for longevity, they are great at getting data out fast. Combine this with a server, or a data hosting/streaming service and you have some type of redundancy. Always make sure your data is accessible.

So now you might want also want to save physical media. This can be as easy or as difficult as you want it to be. Saving your physical media is best done by making it digital. The shelf life of digital data is only going up these days while physical media like paper or tape only degrades. While I’m a fan of my physical media, transferring it to a digital format is the best way to share it and keep it alive.

When dealing with publications or photographs, you can generally get good results digitizing them with a decent scanner. I happen to be a fan of Epsons, but anything around $100 these days should be able to give you decent quality scans. As always, read the reviews just to make sure. If you’re feeling a bit more crafty, you can try your hand at creating a book scanning set-up. This can easily copy all of your publications quickly. After you make your scans, you can perform any type of compression you wish, and even take the resulting image files to assemble a PDF.

Video can also be challenging to save. Whether it be VHS, Laserdisc, or even DVDs, things can get messy. When dealing with your older analog media, you can find devices to digitize them. For example, you can get a capture card like a Canopis ADVC, or any number of DVD recorders on the market. There are also a slew of other little gadgets to clean up the video along the digitizing process. After you get the video converted, you can compress the raw capture down with a codec such as h264 to make the file more manageable.

Audio can be viewed in a similar fashion to video. You can easily pipe a tape or record player into a receiver and feed the output to a nicer sound card. Here, audio can be captured using a program like Audacity and saved as a lossless file or compressed with something like Vorbis or MP3.

With something to drive you, a little know-how, and a lot of time, you can easily start archiving the media in your world. Though it may be daunting at first, you can easily build as you go. Start small, and end up saving big.


Organize Your Data, It’s Going To Be Mine One Day

This article was originally written for and published at The New Tech on June 8th, 2012. It was a collaboration between Moonlit and myself. Enjoy 🙂

Famicoman –

I think I’ve always been an archivist. A vital ally in the digital world. I’m the guy that saves a file from six years ago and pulls it up when people wonder whatever happened to it. I’m the guy who is going to make sure you can still find The New Tech episodes in 20 years, whether anyone would want to or not.

Some might call me a hoarder. Technically, by definition, they are correct. But just like how the word “hacker” has been usurped and manipulated by mass media, so has this term. The word conjures up television-tinted images of people living in trash and debris. It isn’t always like that. Things I save are organized, studied, and shared with the world, not rotting away in some closed off building. Not sealed from the world. If anything, I save because these items may be important to someone else. I’m not always part of the equation.

One could argue that you’re born with an archivist instinct. My philosophy has always been that to be able to look forward, we must look back. Besides digital data, I collect physical artifacts of our technological past. You can learn a lot about Blu-ray by looking at Betamax. This resonates in all archiving. There will always be someone wanting to know how we got to where we are, and hopefully he isn’t left with puzzled faces.

My digital archiving habits started with the world of internet video. In the beginning, I was maxing out my DSL connection and throwing videos up on to Google Video. That later evolved to the IPTV Archive and ultimately my current efforts with archiving Revision3 and a wider range of digital content.

Archiving isn’t an easy task. It isn’t just plucking files off of a download page. It’s mastering wget. It’s manipulating URLs. It’s fighting tooth and nail with a server for weeks, months. It’s talking to people, some of whom don’t want to be talked to. It really stops becoming a hobby and starts being a mindset. You begin to look at things differently, communicate differently, prioritize differently.

When I started out with the IPTV Archive, things were simpler. I could just go download episodes from show sites and be on my way. Now, I get to sites that don’t want to be downloaded in their entirety, and are definitely not set up to be. For example, last year I worked on backing up portions of After a while, they’d lock me out of their servers and the only way to keep downloading was to get a new ip address or wait the block out. This year with Revision3, their CDN throttles me, which ultimately just means I’m going to be waiting longer for their files. For whatever reason, corporations are not fans of someone downloading their entire library of material. Some entities are set up with commercial content, meaning eyeballs are numbers. If you mirror their content, they don’t get as many viewers and less viewers mean less money. In this light, I’m an enemy. I’m a thief. More importantly, I’m a necessity. Without me and those like me, entire cultures could be snuffed out like a flame. Many already have. It’s a strange feeling when you’re contacted by a show creator asking if he could download his episodes from you.

Archiving someone’s digital work is a weird concept to get your head around. Think if you were approached and someone wanted a copy of your entire website. Every little detail becomes theirs to thumb through, spread to others, and replicate for years after you’ve brought the original down. It’s weird, but it’s necessary. When someone years down the road says, “Man, I wish I could watch some old Revision3,” I’ll be there to say, “Here is a copy of all their content. Ever. Enjoy.”

It would be wonderful if it was all as easy as hitting a button and someone’s site downloads for you, but it’s never that simple. Most websites are not designed to be cloned so readily. They lack internal organization. When you peel back the layers, you’d be surprised to see how clumsily some large sites are maintained and held together with rubber bands and paper clips. Out of convenience, we can pull up the Revision3 example again. So many episodes are mislabeled, so many links are dead, the formats for each episode can vary at will, and there are so many episodes and full shows that are just outright gone to the point that if you had no prior knowledge, you wouldn’t know them to have ever existed. It feels like someone ripping pages out of a book and passing it off as if nothing happened.

You have to be one part resources, one part nice guy, one part detective, one part historian and one part hacker. You have to learn about the missing files, you have to track them down, you have to communicate with others who may have them, you have to have the storage and bandwidth to get them, and you have to do it all no matter what is trying to stop you. You have to do all these things, be all these people, at the same time. Sometimes, you have to do it as quickly as possible.

After you gather everything, there is always the question of how to preserve it and disperse it. You have to keep the files up, and make sure they’ll stay up. More importantly, you need to make sure that people can get to them without jumping through hoops. I’ve tried everything on this front. Torrent sites, ftp drops, streaming services, etc. but have ultimately cemented my toolbox with For the uninitiated, the Internet Archive is a non-profit digital library offering permanent data storage. It’s big, and it’s growing every day. Anyone can upload content provided it’s licensed to be distributed openly. It makes things easy when I can be bringing things in through the front door, and flipping them right out the back to

Digital archiving is a brutal but rewarding process that most people don’t see on the front lines. The next time you’re going to put something up online, take a minute to think about it. Your files are going to live much longer than you could imagine. You might as well make it easy for them to.


– Moonlit –

I’ve been a wannabe archivist for some time, but through a mixture of altruistic and less altruistic means, which just so happen to coincide.

On one hand I can’t bear the thought that there is so much recent history that may be, or in some cases already is, needlessly lost forever. Whether it be hardware, software or media, much of what is produced today has no vision for the future, it’s created, it’s used and, ultimately, it’s destined to be lost to whatever forces may eventually whittle its existence down to extinction. Failed storage media, the thought that “if I delete it, somebody else will still have it” or even just plain old waning interest in a flash in the pan which is no longer relevant tomorrow.

On the other hand I find it somewhat distressing that the content I grew up with, much of which came from TV rather than the internet, is very difficult to find. It’s just that little bit too old to have been swept up by a thousand torrent sites or archived to the ever expanding YouTube. It appears to me to exist in a narrow void between content old and popular enough to have made its way to public release via VHS or DVD as a nostalgia trip for the previous generations and the modern piracy scene, who will capture and upload almost anything as pristine digital clones of the broadcast content we enjoy.

Luckily, the two often overlap, so one can be the driving inspiration to accomplish both. But as long as the end result is shared, I don’t view the selfishness of the latter to be a problem. In fact it could very well be a boon, because if everybody was selfish enough to demand copies of the content they thought they’d lost, it means that content still exists, and given that everybody likes different things, meshing all that together would create a patchwork of content from that point in time.

Now, I’ve erred somewhat on the side of piracy so far, but I don’t mean to imply that I’m only interested in commercial media, or indeed in breaking the law. Before moving on though, I’d like to say that I think it’s a collossal shame that in order to capture and preserve certain parts of their lives, we often have to resort to methods which might seem unsavoury to those who disseminate that content. I don’t think it’s unreasonable to suggest that there are indeed large archives these days maintained by large media producers and broadcasters, yet those of us the content was created to be viewed by have no access. Whether that be through music or video clip copyright and licensing issues, laziness or cost, it’s still a great loss to us, and will continue to be until such a time that the content is opened up. This history should not suffer for the sake of a few contracts and a slew of many-digit bank balances. Please, somehow, let this content see the light of day again.

Whew. Got a little bit heavy there. User-created content, there’s a good place to jump to. Podcasts and video podcasts exploded in the mid-2000s along with the proliferation of high speed broadband and cheap consumer cameras. The trouble is, many of those shows had small numbers of fans who, along with the creators themselves, have moved on and left behind their content. This is an important chunk of internet history to me, it got me involved in a large percentage of what I do and who I speak to every day. That’s why I tried my best to help Famicoman build the IPTV Archive when we originally began trying to preserve this stuff. With my pitiful upload speeds and meagre hard drive space, which was frustrating enough, I helped transcode and re-host piles of videos. Those videos were then uploaded to DivX’s Stage6 video hosting site, all neatly encoded in DivX format, with their own special DivX player plugin. Then they took the service down. After countless weeks of pulling down videos, transcoding where necessary, uploading back to Stage6, straining my resources as I went, it was all for naught. Once bitten, twice shy, as they say, and since then I’ve been very wary of trying to do it again, but I’m slowly getting back on the horse. Lesson learned: redundancy. Redundancy and backups. Everywhere. Never rely on any single service to host this kind of stuff, it might be gone tomorrow.

Things get a bit weird somewhere in the middle of those two areas of content, though, with companies like Revision3. They began as a show, or later a couple of shows, which very much fit into the user created content model, a couple of guys with a camera drinking and talking out of their arses for 20-30 minutes. But then it changed. It became the Revision3 we have today, the corporate ad-driven sludge that could very well have been taken direct from the TV and uploaded wholesale to the internet. I’m not against making a profit on content, but stop sucking the soul out of it, it feels like it’s hurting the product. But I’m not here to rag on content creators, my point here is that no matter how poor, tasteless or boring I believe the content or its presentation to be, it still deserves to be archived. What’s crap for me might be gold to somebody else, and it’s not my job to curate history in the making. If I even began to try I would doubtlessly decide that something which later turned out to be pivotal in the future was actually the naffest thing to ever grace a visual display. I believe Jason Scott made a similar point about the preservation of GeoCities. Yes, it might be full of weakly written, poorly laid out, eye-damaging animated horribleness, but it’s historical weakly written, poorly laid out, eye-damaging animated horribleness. It’s a snapshot of what the internet was at that time, and as such it should not be forgotten. So go forth and grab it, grab it all, because as hard as it might be to believe, one day it will all be gone.