Mining Bitcoin for Fun and (Basically No) Profit, Part 3: Mobile Development


If you have not done so already, please read parts 1 and 2 in this series.

So I have a mining rig that’s successfully rewarding me with bitcoins. Normal people would probably stop at this point. One nice thing about mining in Slush’s Pool is that it has a handy email notification option that tells you when credit is being transferred to your Bitcoin wallet. This is pretty cool, but what if I want more in-depth information? For example, what if I want to know my hash rate, or if my miner is alive (did the system crash?) or how many bitcoins I have total?

The next step for me was to create a mobile application which could provide all this information – whenever or wherever I wanted it. So, I got to work.

The platform I chose to work with was Android. A logical choice for me as I had prior experience developing Android applications and own an Android phone myself. Programming for Android, as many know, means programming in Java. If you have any prior Java experience, you’re already have a head start if you ever wanted to get into Android development.

A fantastic thing about Slush’s Pool is that it offers an API (Application Programming Interface) which allows users to pull down information on their miners using the de facto JSON format. So from this I can get at my mining information, but what else do I want? I decided it would be wise to pull down the average value of a bitcoin in USD, at any given moment. This way, I can do some simple calculations to determine a rough estimate of how much I’m generating and getting payed in USD. Lastly, I wanted to get the balance of my Bitcoin wallet, again to be displayed in both Bitcoin and USD.

I already had the API information for Slush’s Pool, as it is linked on everyone’s profile and accessed via a common base url and unique key for each user. Here is an example of the JSON output for my account:


{
    username: "Famicoman",'
    rating: "none",
    confirmed_nmc_reward: "0.00000000",
    send_threshold: "0.01000000",
    nmc_send_threshold: "1.00000000",
    confirmed_reward: "0.00145923",
    workers: {
        Famicoman.worker1: {
            last_share: 1378319704,
            score: "70687.6038",
            hashrate: 1004,
            shares: 1906,
            alive: true
        }
    },
    wallet: "1DVLNHpcoAso6rvisCnVQbCFN8dRir1GVQ",
    unconfirmed_nmc_reward: "0.00000000",
    unconfirmed_reward: "0.00612688",
    estimated_reward: "0.00046390",
    hashrate: "1006.437"
}

Next, I needed an API for the average value of a bitcoin in USD. I went on the hunt. Finally, I found that Mt. Gox, the largest Bitcoin exchange, has a public API for bitcoin rates (located at this ticker URL). This works perfectly for my needs. Here is some sample JSON output from this API:


{
    result: "success",
    return: {
        avg: {
        value: "143.97798",
        value_int: "14397798",
        display: "$143.98",
        display_short: "$143.98",
        currency: "USD"
        }
    }
}

So far, so good.

Lastly, I wanted wallet information. I discovered that Blockchain shows records of transactions (as they are all recorded in the block chain), so I did some probing and found they also offered an API for attributes of individual wallets (Here’s a link using my wallet info, it’s all public anyway). This includes balance, transactions, etc. The units for bitcoins here is the Satoshi, one millionth of a bitcoin. Some sample JSON output from this service looks like so:


{
    hash160: "88fd52ba00f9aa29003cfc88882a3a3b69bfd377",
    address: "1DVLNHpcoAso6rvisCnVQbCFN8dRir1GVQ",
    n_tx: 7,
    total_received: 8434869,
    total_sent: 0,
    final_balance: 8434869,
}

So now I had the APIs I was going to use and needed to put them all together in a neat package. The resulting Android application is a simple one. Two screens (home and statistics) with a refresh button that pulls everything down again and recalculates any necessary currency conversions. Android does not allow you to do anything system-intensive on the main (UI) thread anymore, so I had to resort to using an asynchronous task that spawns a new thread. This thread is where I pull down all the JSON (in text form) and get my hands dirty manipulating the data. I utilize a 3rd party library called GSON to parse the data I need from the JSON string. Then, it’s just a little bit of math and we have all the necessary data. After all of that is done, the application prints everything on the screen. Pretty basic, and with plenty of room for potential additions.

When running the application, provided there’s network connectivity and all the servers are up, you will be rewarded with a screen like this:

The app in action

The application in action

Not too shabby. If you wanted to use it yourself, it would be necessary to hard-code your own key from Slush’s pool. There doesn’t appear to be an API call by username (by design), so it needs to be implemented manually at some point (which happens to be in code as of right now).

The source for this application, which I call SlushPuppy, is freely available on GitHub. Feel free to fork it, or just download and mess around with it. If anything, it provides a small example of both Android-specific programming as well as API interaction.

Tags: , , , , , ,

Mining Bitcoin for Fun and (Basically No) Profit, Part 2: The Project


If you have not yet, please read the first article in this series: Mining Bitcoin for Fun and (Basically No) Profit, Part 1: Introduction

I have a few Raspberry Pis around my house that I like to play with – four in total. Prior to this idea of a Bitcoin project, I had one running as a media center and another operating as a PBX. Of the remaining two, one was an early model B with 256MB of RAM while the other was the shiny new revision sporting 512MB. I wanted to save the revised model for the possibility of a MAME project, so I decided to put the other, older one to work. It would help me on my quest to mine bitcoins.

Raspberry Pi Model B

Raspberry Pi Model B

But what is mining exactly you may ask? Bitcoin works on a system of verified transactions achieved through a distributed consensus (a mouthful I know). Every transaction is kept as a record on the Bitcoin block chain. Mining is more or less the process of verifying a “block” of these transactions and appending them to the chain. These blocks have to fit strict cryptographic rules before they can be appended, else blocks could be modified and invalidated. When someone properly generates one of these blocks, the system pays them in a certain amount of bitcoins (currently 25). This process repeats every ten minutes.

I knew that at this stage in the mining game I had to go with an ASIC setup and I new I wanted to run it off of my Raspberry Pi. Simple enough. The Raspberry Pi is a fantastic platform for this considering its price, power consumption, and horsepower. For mining hardware, I decided to buy the cheapest ASIC miners I could get my hands on. I found the ASICminer USB Block Eruptor Sapphire for the low price of $45 on Amazon. They cost more money on eBay and I couldn’t buy them from any sellers with Bitcoin because I didn’t have any (and didn’t want to bother with exchanges) so this seemed like the way to go. The Block Eruptor could run at ~330MHash/s, which is pretty hefty compared to GPU mining and at a fraction of the price. It is also pretty low power, using only 2.5 watts.

ASICminer USB Block Eruptor

ASICminer USB Block Eruptor

So I figured that I would get one of those, but also devised a more complete and formal parts list:

  • 1 x Raspberry Pi
  • 1 x ~4GB SD Card
  • 1 x Micro USB Cable
  • 1 x Network Cable
  • 1 x Powered USB HUB
  • n x USB Block Eruptor

That’s the basics of it. I already had the Raspberry Pi, and the necessary cables and SD card. These were just lying around. I needed to purchase a USB hub, so I bought a 7-port model for about $20. The hub needs to be powered as it will be running both the Raspberry Pi and the USB Block Eruptor. Considering power, the Raspberry Pi claims to draw somewhere around 1-1.2 Amps maximum while the USB Block Eruptor claims to draw 500 milliAmps maximum. I tested things out using my Kill A Watt and found that my setup with the USB hub, Raspberry Pi, and three USB Block Eruptors draws only 170 milliAmps and uses only 12.5 watts! So the projected power usage seems off for me, but I can’t guarantee the same results for you.

After buying my USB Block Eruptor on Amazon, I got it in about a week. The day after I got it and made sure it was working, I ordered two additional units to fill out the hub a little more.

Software

To do anything with Bitcoin, we’re first going to need a wallet and a Bitcoin address. Which wallet software you use is up to you. For desktop apps, there is the original Bitcoin-qt, MultiBit, and other third-party wallets. There are mobile applications like Bitcoin Wallet for Android, and even web-based wallets like BlockChain that store your bitcoins online. Figure out what client works best for you and use it to generate your Bitcoin address. The address is a series of alphanumeric characters that act as a public key for anyone to send bitcoins to. This address is also linked to a private key, not meant to be distributed, which allows the address holder to transfer funds.

In order to use the USB Block Eruptor, we’re going to need mining software. One great thing about using the Raspberry Pi as a platform is that someone has already made a Bitcoin mining operating system called MinePeon, built on Arch Linux. The distribution combines existing mining packages cgminer and BFGminer with a web-based GUI and some nice statistical elements. You’re going to need to download this.

To copy the operating system image file onto the SD card for your Raspberry Pi, insert the card into your computer and format it with the application of your choosing. Since I did this with a Windows system, I used Win32 Disk Imager. It is fairly straight forward: choose the image file, choose the drive letter, hit write, and you’re done.

Win32 Disk Imager

Win32 Disk Imager

Okay, software is ready. Now to set up the Raspberry Pi, insert the SD card into the Pi’s slot. To power the Pi, plug your Raspberry Pi into one of the USB ports via Micro USB cable. Then, plug the USB hub into one of your Raspberry Pi’s free USB ports. Any USB Block Eruptors you have can be plugged into the remaining USB ports on the hub (not the Raspberry Pi directly), but keep heat flow in mind as they get pretty hot. Next, connect the Raspberry Pi to your home network. Finally, plug your hub’s power cord in and let the Pi boot up.

To go any further, you will need to determine your Raspberry Pi’s IP address. If your router allows for it, the easiest way is to log in to it and look for the new device on your network list. Alternatively, plug a monitor or television into your Raspberry Pi and log directly into the system using minepeon as the user and peon as the password. From there, run the ifconfig command to retrieve the internal IP address.

Navigate to MinePeon’s web interface by typing the IP address in your browser. You’ll have to log in to the web interface using the previously defined credentials: minepeon as the user and peon as the password. You should be presented with a screen similar to this (But without the graphs filled in):

MinePeon Home Screen

MinePeon Status Screen

If you receive an in-line error about the graphs not being found, don’t worry. You just need to get mining and they will generate automatically, making the message go away.

Configuration

In order to utilize the software, you will now need to register with one or more Bitcoin mining pools. Mining pools work using distribution. The pool you are connected to will track the progress of each user’s attempt at solving a block for the block chain. On proof of an attempt at solving the next block, the user is awarded a share. At the end of the round, any winnings are divided among users based on how much power (how many shares) they contribute. Why use a mining pool at all? Payout usually only happens for one user when a block is solved. It can be very difficult for a user to mine a bitcoin, even after months of trying as the odds of success are always the same. Mining independently offers the opportunity of a giant payout at some point, but pooled mining offers smaller, more regular payouts.

Mining pools can differ greatly in how the payout is divided. I’d advise that you do some research as to which method works best for you. Alternatively, you could go about setting up your own mining pool, but I wouldn’t advise it without a substantial amount of processing power unless you’re willing to wait for a payout (if it even comes at all).

Anyway, I chose two mining pools: Slush’s pool (my primary) and BTC Guild (my fail-over in case my primary is down).

Registering with a mining pool is as simple as registering with any other website. After completing registration, you will be supplied with a worker name/password and the server address. These credentials can then be pushed into the MinePeon Pools configuration like so:

MinePeon Pool Settings

MinePeon Mining Pools

Next go to the Settings page and change your password for the MinePeon web interface (it’s a good idea), the timezone (this is buggy right now and won’t look like it’s working on the settings page) and any time you want to donate to the MinePeon maintainer (if any).

MinePeon Settings

MinePeon Settings

If everything is configured correctly, within a few hours (or instantly), you should see some activity on your MinePeon Status screen. Additionally, be sure to check your account on the mining pool you signed up with to make sure everything is working as expected.

My MinePeon Pool & Device Status

My MinePeon Pool & Device Status

My Slush's Pool Worker Status

My Slush’s Pool Worker Status

Now, just sit back and let your machine go to town. The only thing you have to do at this point is make sure the USB hub continues to get power (don’t let anyone unplug it) and it should run continuously. On your first day or two, it may take a while before your status update and payout will take even longer.

My mining rig, hub side

My mining rig, hub side

My mining rig, Raspberry Pi side

My mining rig, Raspberry Pi side

Most mining pools offer status tracking for your payout, so you should be able to see how things are progressing fairly quickly. As of this article being published, I receive a payout of 0.01 Bitcoin near every 24-30 hours while running three USB Block Eruptors.

Tags: , , , , , , , ,

Mining Bitcoin for Fun and (Basically No) Profit, Part 1: Introduction


Note: This article is the first entry in a series I am writing for Philly2600.

If you’re anything like myself, you’ve been keeping loose tabs on Bitcoin over the years. When I first read about the cryptocurrency, I thought it was an awesome concept. Now, I had heard about electronic currencies before. The first mental link I made upon hearing about Bitcoin was that it reminded me of e-gold. Founded in 1996, years before Paypal, e-gold was a gold-backed digital currency created by a few guys in Florida. e-gold was the de facto currency for underground transactions, and was recently referenced in Kevin Poulsen’s Kingpin as the choice of the carder market – the collection of online outlets to buy and sell credit card information.

egold's logo

egold’s logo

Though it was forged near the tail of e-gold’s run and adopted a similar concept, Bitcoin turned out to be a different beast entirely. While it is still favored for underground transactions, closely integrated with controversial websites like The Silk Road, the currency had striking differences that allowed it to come into its own. Bitcoin is distributed (read peer-to-peer), decentralized, and considered fiat money (as opposed to representative money). There is no central authority to go to with legal matters, you cannot simply flick a switch and shut down the network, and the currency only has value because we give it value – it isn’t backed against gold or silver or another currency.

Bitcoin also has an interesting history. The identity of the creator, who goes by the name Satoshi Nakamoto, is still unknown to the public. Many theories have come up to who the man behind Bitcoin really is. Some speculations range from an academic team to a government agency to a reclusive cryptographer. If you want to see more speculation, there’s an interesting Vice article about the whole thing I’d recommend checking out.

Background

In 2011, I got interested enough in Bitcoin to set up my own wallet, download the block-chain, and set up little donate buttons on a blog or two. The donations never rolled in (and why would they), but my fascination with the technology did. The idea of a monetary system that worked sort of “like BitTorrent” not only held my attention because of the possibility of financial success but also because it made me feel like I was at the forefront of something cool and exciting. I pictured scenes straight out of Serial Experiments Lain or Neuromancer with a dingy apartment somewhere in a dense city. A patchwork of tangled computer cables linking unknown and mysterious hardware together to just run and create money for me while I’m out. Nothing ever sounded both so cyberpunk and actually possible (though probably not as bleakly artistic).

At the time, CPU mining was on its way out as GPU mining was taking over. The internet was flooding with pictures of enthusiasts’ mining rigs. Case-less computers, motherboards with a large amount of PCI slots, each filled with a top-of-the-line graphics card. One of these setups was big, hot, messy, expensive, and beautiful. Usually a person would have a few of these chaotic mining machines all running in the same room and they caught the cyberpunk feel I so badly wanted to create for myself. I wanted the hectic rat’s nest of wires and satisfaction of a successful rig build.

The "Super Rig"

The “Super Rig”

I never got that far. Building a machine to do this was an expensive process and I didn’t want to put a huge investment on the line when I was operating on a limited budget in the first place. So, ultimately, I steered away from mining as a whole.

That didn’t turn me off from the whole technological concept though. I did end up surveying the field to see what people were using Bitcoin for. The possibilities seemed endless. Aside from sales of underground goods, I saw there was gambling, web hosting, and even retails sales (including some coffee shops). Pretty much any type of business that could accept Bitcoin was starting to have outlets that accepted the digital currency. I did what any Bitcoin novice did: got my 0.005 BTC from BitFountain for free (now defunct), and sat on it. No use doing anything with it. One bitcoin was worth around $8 USD at the time, so I had about four cents.

After the GPU mining wave, I next saw the FPGA generation. FPGA stands for Field Programmable Gate Array and is pretty much self-explanatory. An FPGA has a hardware array of logic gates like your typical AND or XOR operations. Sequences of logic gates can be put together to form half-adders and multiplexers and eventually processors (when you chain enough smaller components together). Normally, you would have all of these components pre-determined into some type of integrated circuit called an ASIC (standing for Application Specific Integrated Circuit) which are designed and programmed only for certain unique tasks. Think of an FPGA as a breadboard for the final ASIC design. Both the FPGA and ASIC are programmed in an HDL (hardware description language) such as VHDL or Verilog (or any other ones you might remember from a System Architecture class). Unlike your typical object-oriented or scripting languages, an HDL is more suited for the Electrical Engineer instead of the Software Engineer (me). An HDL allows you to create models and interactions of hardware components as though you had them available physically.

XILINX Spartan-3E FPGA

XILINX Spartan-3E FPGA

As you’d guess, FPGAs were a favorite for Bitcoin mining enthusiasts. Developers would program the boards to mine Bitcoin and leave behind anyone still pushing their GPUs to the limit. For me, FPGAs were still a massive investment. Though likely not as much as an outfit of new GPUs, coupled with enough electricity to power a small town, the amount of money for an FPGA was still a few hundred dollars. On top of that, I’d still have to dust off some of my class notes and program the thing. It would have been fun and a great learning experience but at the time I didn’t want the hastle. Besides, something better was coming soon anyway.

In the summer of 2012, I started discussing with my co-workers the feasibility of having us set up a Bitcoin mining operation. The whole concept was relatively simple: we were all going to throw money in for a new USB connected ASIC chip and run it off of a computer of our own. We did the math to figure out power consumption, our initial investment, mining complexity increase, etc. and the numbers for our break-even point looked pretty good. The company were were looking at for our miner was Butterfly Labs, who boasted they could provide a chip with an incredible hash rate at only a few hundred dollars. Split between a few people, it didn’t seem like too bad of a deal. Then, we started looking into the company. They were plagued with manufacturing delays. When you couple the time delay with the growing mining complexity, your return takes much longer. Couple that with the fact that Butterfly hadn’t delivered anything yet, the whole thing could have been someone’s pie-in-the-sky idea or giant scam. We decided to shut down our little plan and save ourselves the aggravation. This ended up being a wise decision. Butterfly Labs continued to be plagued by delays and people ended up auctioning off their pre-orders. There are still not that many Butterfly Labs ASIC chips out in the wild, even now.

Butterfly Labs ASIC Miner

Butterfly Labs ASIC Miner

After all this, I still wanted to try my hand at Bitcoin mining.

I knew that I wasn’t going to make a lot of money, but I thought it would be fun. If it made me any money, any at all, that would be something. So I got to work doing a little research.

Tags: , , , , , , ,

The Summer Backlog


Every Summer I speculate that I’m going to have an unbelievable amount free time. It will always be so fantastic and freeing. I’ll be done school, working a stress-free job, and there will be so much unscheduled time that I’ll just get bored and come up with hundreds of new tasks for myself.

This never happens.

Well, the having-free-time-thing never happens but I do take on new activities anyway. After enough time, I end up with a bunch of things I’ve been meaning to do, and work on them impulsively at sporadic intervals. Everything moves forward, slow and steady, but in an agonizing and chaotic fashion.

I do make time for my projects, but the available time is fluctuating as the years go by. When I started these projects all I had was time and energy, but no money. Now, I seem to find myself with a modest amount of money and energy, but no time. Eventually, I’m doomed to have time and money, but no energy. This is the vicious cycle, and here I find myself in the second stage.

Without organization, every project falls on its face. I’m a big proponent of organization, especially when I have so much going on. After a while, you just need to keep track and work smarter (or risk meeting some men who want to put you in a straitjacket). Below, I’ve outlined (to the best of my ability) the various projects I’m working on, and where they need to go next. Hopefully this not only helps me stay on track but also gives you something to yell at me about the next time you see me.

Obsoleet
I’ve actually had most of an episode filmed for a long time by this point. The only problem that I faced was the audio cut out at the end of one of the shots. After I redo it, which I wanted to do anyway, the footage should be mostly set to go into editing. Additionally, I’d like to film a little skit for the intro if I can manage it. Editing usually doesn’t take a whole lot of time, though I do want to try out some new software and I have to cut a brand new introduction. High definition video also proves to be more of a hassle and take some more (read unplanned) time.

Anarchivism
This one is going along pretty well, especially recently. On the scanning side of things, I have plenty of stuff coming in but not a lot going up. The scanner I have is awful when it comes down to conducting magazine scans and I’ll have to look for something beefier before going full tilt on my library. As an aside, I’ve more or less created the most complete wiki of hacker magazines complete with information on them as far as I can tell. With my current rig, I can pump out some more Blacklisted! 411 issues without much hesitation.

Going after Revision3 has slowed a little, but I can get back into it with some one-liners soon. Getting to other odds and ends comes and goes as I find them. The only section that could have hours poured into it is the hacker con category. The videos I find not only have different ways of being obtained but also get updated with a new crop annually, so everything is constantly in flux. I’m trying to hunt down some of the more difficult stuff as well as fill in actual information about the conferences. If you want to help out, please do.

TechTat
This one is more or less dead due to lack of interest. While it was cool having a collaboration site for retro tech, it lost its luster after a few months. I considered turning TechTat into an audio podcast but I’m not sure how that would turn out. I’m certain I can find some use for the concept.

ChannelEM
ChannelEM keeps trucking on, but is prone to frequent crashing. It does seem to get more stable after software updates, but still ultimately hangs. I want to take a look at the scripting done to run the station and see if I can put in any fail-safes to stop the crashing. CEM also needs a rotation update with any new episodes. Further, the idea of getting new shows to join up is a bit fruitless now, but the site does well as it stands. For no real reason at all, I’d like to see if I can add on to the existing scripts and create a JSON API with scheduling information.

Moonlit has also been working on some very interesting video projects that I’d like to integrate which would completely change the look and feel of both the site and the content.

Raunchy Taco
More or less in a standstill. The stability fluctuates and there isn’t that much going on there anyway. The IRC server is really only kept up if Ethan, Pat, and myself need a place to chat. For a network that has been off-and-on for 6-7 years, we have empty periods like this all the time. I’d like to just keep it up if I can.

The IPTV Archive
More or less in waiting. I put up a hefty amount of content, and then ultimately mirrored it to Internet Archive where it can live forever. If I had the time, I’d spend it doing more detective work for the missing shows- there is always more detective work to do. There are probably a half dozen more smaller shows I could throw up at some point but nothing too pressing.

Additionally, when I started the site I used Blip because it had (arguably) the best quality at the time. Now, YouTube has eclipsed it. There was a bit of panic a few months back about some Blip channels being closed down for no reason and I have to entertain the idea that this could happen to me. If that happens, the whole library would likely need to be moved to YouTube. A big move, but likely a nice one for the content.

Moreover, I’ve also considered moving the content over to Anarchivism as it would be a much more flexible platform.

House Keeping and Solo Projects
I enjoy writing and I’d to do more of it. Besides just being more active here, I’d like to get back into writing for other outlets. I’m thinking of more for The New Tech, and another for my local 2600 group. I’ve also been playing around with Medium (I like the concept but it still might be pretentious dribble) and would like to publish another article through it. I’m looking into 2-3 print publications as well if I can come up with the right topics and go into those pieces with the right energy.

Aside from my web work, I have a bunch of little, lower-profile things going on that I need to get out of the way.

I recently got a display for my Apple G5, so I can let it run as a capture PC for video transfers. I already have an ADVC box hooked up and the machine captures great… but it needs a monitor hooked up to run. Then, I can do more video transfers which can ultimately pop up in other places (Maybe a found footage section on Anarchivism).

I want to set up a dedicated headless Linux server for staging web projects amongst other things. I might also have it just run wget scripts all day or some custom web crawlers or who knows what else.

I have an old cocktail arcade cabinet that needs some love. If the original electronics are beyond repair, it would be nice to outfit the cab with new hardware and set up a MAME machine.

More Raspberry Pi projects would be nice. I like having the Incredible Pi set up as a PBX but I feel like I could do more with it. I have another Pi set up as a media center that I use often. I’m currently on setting up a Bitcoin mining rig with another and still have many more ideas. Raspberry Pi cluster? Telnet BBS? BBS hooked into the PBX? The possibilities are endless.

Paranoia kicks in with regard to my data. I have a dozen or so terabytes worth and I need to clean data off of old drives, sort it, duplicate it, and duplicate the data that’s already there. To make matters worse, I’m constantly downloading more.

A CJDNS Meshnet node has also been in the works for a long time. I tried to set up my first one on a PogoPlug and while I eventually got the software to compile, I couldn’t connect to anybody. It may be time for another try, and possibly on a “normal” box before adapting it to the PogoPlug.

I’m experimenting with a few more programming languages and development environments. Recently, I’ve looked into running some Go, and am learning a great deal of JavaScript. I’d like to look into C# and also play with the Unity engine. Aside from these, I’m reasonably proficient at Android development and might be tying this in with another project of a friend’s.

And the list goes on.

 

There’s a lot of things here- a hell of a lot of things. I’d be lying if I said that there weren’t going to be even more. Hopefully, as I now have a nice little outline, I’ll be able to zero-in my focus and get some work done.

In the meantime, sit back and enjoy the show.

Tags: , , , , , , , , , , , , , , , , , , , , , , , ,

What Administrating a BitTorrent Site Taught Me About Project Management


This article was originally written for and published at Medium on May 18th, 2013. It has been posted here for safe keeping.

In my sophomore year of college, I became an administrator of a BitTorrent website. It’s not nearly as shady as it sounds. In fact, it was a small and completely legal operation. Three administrators, one server, and hard drive after hard drive full of Creative Commons-licensed content.

Now, I’m lucky enough to attend an undergraduate school with a strong internship tie-in. We spend half of the year slaving away on our school work while the other half is spent in one of those real-world jobs. Lather, rinse, repeat for three years. On the academic side, we take our specialized engineering classes, our project management classes, our technical communication classes, our how-to-work-with people classes.

I didn’t take many of those yet. They first year and change at the university is mostly populated with weed-out classes and introductory curriculum akin to a secondary school elective class or two. At this point in my life,I didn’t know what makes a good project and I didn’t know what makes a project good. I didn’t know how to communicate effectively or work as part of a group. I didn’t know about Gantt charts, or deliverables, or development practices.

As I mentioned, there were three of us. One administrator I had met via online chat some months prior in a public channel. He was a decent guy, and the linkage between myself and the mysterious third administrator who I had never spoken with but was providing us with a server. We all came together, communicating with each other in a strictly online format. Geographically separated, what did it matter with email and a few common hours when we all happened to be awake at the same time? We didn’t have structure or a real thought-out plan. No documents or task lists or meetings to touch base. We carved out and constructed bits and pieces when we felt like it and waited for each other to catch up before charging forward again full steam.

It happened to be winter break, and I had plenty of free time to devote. After we eventually got the site up and operational, I spent days filling it with uploads and tutorials, configuring and reworking plugins and style sheets, setting up social networking accounts, and more or less doing my damnedest to make it ready for prime time. Then, we got a pay-off. A file-sharing blog picked up on the site and did a piece. Within 48 hours, news spread and we had some 3,000 members. We were being reblogged and discussed in forums. We were growing by the hour.

Sounds great, huh? It wasn’t.

While we had all been united in our quest to launch a fantastic niche torrent site, we quickly split at the seams. While I tried my best to keep a steady flow of content being uploaded to seed the site for new users, the other admins didn’t seem as compelled to. One simply disappeared for weeks at a time while another decided it would be a good time to ask for donations and not do much else. Our chat sessions together got shorter and eventually vanished completely. The site stagnated except for a small group of hopefuls that were uploading and contributing, but it amounted to too little. We fell apart. We were broken.

One day, I made a passing comment to a user about how I’d like to rebuild and relaunch the site, and then found myself stripped of my administrative permissions. I contacted the one administrator I had known prior to starting the project, and he just shrugged off the situation as weird before reinstating me into the ranks. It was too late, though. I ended up deleting myself of my own accord a week later.

I completely removed myself from the project, but that doesn’t mean I left empty-handed. I departed with lessons forged from mistakes and successes. What worked, and what didn’t. I learned the need for defining a project scope and keeping open the lines of communication. I learned the importance of meeting regularly and setting goals and being assertive. I learned sacrifice and when to cut your losses and move on.

Each one of these lessons followed me as I went from internship to internship and class project to class project. Academia can teach you a good amount about how to be a developer, but falls a little short when it comes to how to work with real people in the real world.

To learn that, well—you just need to experience it.

Tags: , ,

Films to Look Forward Too


Just a follow-up to the last article with some new films. It’s been pretty exciting as some of the ones on the last list started rolling out, and hopefully most of these will make it as well. As always, I don’t knows if any of these will be any good, but they’ve captured my attention to the point where I had to make a note of them.

Let me know if I missed any.

Aaron Swartz – The Internet’s Own Boy
Documentary on internet pioneer and activist Aaron Swartz. Anticipated early 2014.

The Archive Documentary
Can’t find a lot of into on this one, but there’s a “part 1″ of it about the Internet Archive (which you can find hosted there for download). Release unknown.

Computer Chess
Interesting fictional film shot on black and white video about 80′s geeks programming chess computers. The technology behind the filming almost interests me more than the plot. Currently screening.

Downloaded
Documentary about digital media and the file-sharing generation. Currently screening.

From Bedrooms to Billions
Documentary on the video game pioneers located in the UK. Anticipated late 2013.

The Gamer Age (previously Beyond the Game)
Documentary exploring gamer culture from many different angles. Currently screening.

Hackers in Uganda – A Documentary
Documentary about hackers contributing technical education and equipment in Uganda. Currently filming, anticipated early 2014.

High Tech, Low Life
Documentary following two Chinese citizen journalists as they travel the country. Anticipated 2013.

Inside the Dragon’s Lair
Documentary focusing on the legacy and history of the groundbreaking LaserDisc-based ’80s arcade game. Currently filming.

The Space Invaders: In Search of Lost Time
Documentary about classic arcade game collecting, but also focuses on arcade culture. Recently released.

UNDER THE SMOGBERRY TREES: The True Story of Dr. Demento
Documentary about the legendary DJ, Dr. Demento. Anticipated 2014.

The Video Craze
Documentary focused on ’80s arcade culture. Anticipated late 2013.

Video Games – The Movie
Documentary about the video game industry and culture resulting from it. Anticipated late 2013.

We Steal Secrets: The Story of WikiLeaks
Documentary about WikiLeaks and US government security breaches. Already has a large number of negative reviews. Anticipated 2013.

Tags: ,

Documentaries I’d Like to See Before I Die (Or Everyone Forgets)


I’d like to think that I have something of a second nature when it comes to whether or not there is a documentary made or in production for any of my disjointed hobbies and interests. It’s not one of those skills you showcase in your job interview, but I seem to have this knack for religiously crawling the web in search for films I think I’d enjoy. Surprisingly, and to my great pleasure, a lot of these fringe interests I posses already have films about them. Awesome. However, there are a few that simply do not- or, have a film that doesn’t satiate my particular appetite.

So, for my sanity, I made a list of the topics I’d personally like to see filmed. And, in some cases, some topics I’d probably find gratification in filming myself.

Written below is that very list. Think of this more as a way of me getting the thoughts from my head to paper as opposed to a list of full-bodied explanations and fleshed-out ideas.

Demoscene. There are already a few demoscene documentaties out there. For example, The Demoscene Documentary is about the demoscene in Finland and Moleman 2 is a demoscene documentary focusing mainly on Hungary. While these are in fact good films, they each have a specific scope. From what I gather, the demoscene can be radically different from country to country, making it difficult to understand as a whole when only presented with a few of its parts. I’d propose an episodic piece showcasing the demoscene in a variety of countries – each country having its own segment. While these existing documentaries have touched on Finland and Hungary, there are still Germany, USA, Denmark, and Norway to consider (and probably others).

Bitcoin & Digital Currency. We’ve all heard of Bitcoin by now, especially as it makes waves at it’s current high value. However, Bitcoin itself has an interesting past and makes an interesting statement. If you do any detective work about how Bitcoin came to be, you will be sucked up into a mysterious story about how nobody knows the identity of the creator or what happened to him. The conspiracy theories are vast and plenty. We also touch on the interesting issue of an unregulated worldwide currency, governments attempting regulation, bitcoin-mining malware botnets, attacks on exchanges, etc. How about how crazy some people go with their mining setups? Dozens of caseless computers fillied with graphics cards- a cyberpunk daydream turned reality. How about using FPGAs and these new ASIC rigs? Now, that’s just bitcoin. There are numerous other digital currencies out there such as the newer litecoin, or even e-gold (Created in 1996). Digital currency has been around longer than most people think.

Cypherpunk. The cypherpunk movement does for cryptograhy what the cyberpunk scene did for personal computing. While cypherpunks have been around for decades, the interest within the scene has been renewed and pushed towards the mainstream more recently. Going back to “A Cypherpunk Manifesto” and the cypherpunk mailing list, we see early discussions of online privacy and censorship, paving the way for Bitcoin, Wikileaks, CryptoParty, Tor, 3D-printing of weaponry, etc.

Usenet. Started in 1980, Usenet is a system for users to read and post messages. Usenet can be seen as the precursor to internet forums, and is much like a Bulletin Board System in theory except it is distributed among many servers instead of a central authority. As time goes on, Usenet continues to grow in bandwidth usage, now generating terabytes of traffic a day. This is mostly through binary file transfers as opposed to messages. Despite many main ISPs deciding to remove Usenet access from their internet services, many still seek out paid access.

Pirate Radio UK. While Pirate Radio USA and Making Waves do a fantastic job at covering pirate Radio in the US, I haven’t seen much of an effort to show off pirate radio in the UK. From what I’ve gathered, there are an uncountable number of pirate radio stations across the pond, and it’s a different game when compared to the US. At the peak of pirate radio’s popularity, there were near 600 stations active in the UK while there are presently 150, mostly based in London. Here’s a mini piece from Vice.

Darknet. Not in regards to file sharing. More covering the darknet as a blanket term for an independant or ad-hoc network with some sort of disconnection from the internet. Considering topics like Hyperboria and CJDNS, Tor and the Deep Web, Meshneting for fun or necesity, Tin-Can, and so-on. As the hardware becomes less expensive and more devices have networking abilities, creating a scalable network becomes a more achievable task.

Dyson. I feel that James Dyson doesn’t get as much credit as a revolutionary engineer as he deserves. Dyson focuses on improvement: taking the wheel and making it better. No pun intended, but his first success was the creation of a fiberglass wheelbarrow that used a ball instead of a wheel. Afterwards, he famously created over 1000 prototypes for a new vaccuum cleaner using cyclone technology after noticing problems with his Hoover. Dyson repeatedly uses creative thinking and pulls inspiration from unlikely sources.

Raspberry Pi. While the Raspberry Pi was not necessarilly a unique and new concept, it was certainly one of the most well executed. We have seen other incarnations of plug computers such as the Beagleboard or the Sheevaplug, but the Raspberry Pi’s addition of integrated video sets it apart. And, at the price of $30, makes it incredibly affordable. Many would argue that what makes the Pi so special is the community that has formed around it, and not necessarily the hardware that ties it together. Everyone stretches their imagination and expertise: if it can be on the Pi, it should. Aside from the community, the Raspberry Pi Foundation has been done an incredible job at cultivating the technology and inspiring the next generation of young programmers and hardware hackers.

Kickstarter. There have been documentaries in the works that focus on crowdfunding, but I’m not as interested in the crowdfunding movement as much as I am in Kickstarter the company. While Indie GoGo has been around for a longer time, they do not seem to be held together as tightly. Kickstarter seems like not only an interesting company, but one that holds itself, and those who utilize its services, to a high standard.

QUBE. Here’s an odd one for that likely nobody has heard of. QUBE was the first interactive TV station, started in 1977 in Columbus, Ohio. Residents who subscribed to the cable service received a device that looked something like a calculator that allowed them to communicate back to the station during shows. Aside from the interactive feature, QUBE was on the forefront of pay-per-view programming and special interest content. QUBE soon went bankrupt and dissolved in the early 1980s. As a bit of an aside, I think I actually tried contacting the webmaster of that site a while back to ask if I could get a copy of the “QUBE DVD” for archiving but didn’t get a reply. Let’s hope he/she runs Webalizer or Google Analytics and sees some referrer traffic. Maybe it’ll be enough to spark a conversation.

So here ends my list. While the majority of these ideas are feasible, I can’t help but think a few might end up slipping too far and too fast into obscurity before their time. Other ideas on here might be too early in their lives. Doing something now, or even within the next decade, would only show a small part of the eventual picture.

Do I expect any of these to be made? Not particularly. But you never know.

Everyone gets lucky once in a while.

Tags: , , , , , , , , , , , ,

Saving Rev3 – Update 8 – “Resurrection”


Yeah, I stole the title from the Halloween series. It seemed fitting.

So here we are about a year later. Guess what? Revision3 has killed off a few more shows, started a bunch more, and redesigned their site.

A few days ago, Moonlit and I got into an interesting Twitter conversation with Revision3, the results of which can be found here and here. Basically, with their site redesign they lost a bunch of shows which they then regained after we brought it up. They also claimed to fix some dead links, so some possible good news there. On the negative, they did say that some shows have been removed purposefully. Thankfully I believe I have a copy of everything they’ve taken down to date. On the whole, it felt as though they danced around issues I brought up, but at this point I find it unsurprising. It’s probably a good thing they didn’t Google me while we were talking.

I checked my download script from last summer, and sure enough it didn’t work. Upon doing a few trials, it’s a simple fix to change “small” to “medium.” So, I went ahead and updated it. This now works perfectly again (woohoo!).

I went through the wiki page and added in the shows that they had cancelled while I was out of the loop. I’m just going off of their “Archive Shows” page, which may be incomplete, but I don’t know enough about this age of Revision3 to tell you if something is missing or not. It doesn’t look too much as if they throw out shows altogether these days.

I count 10 more dead shows.

 

On a similar topic, I’m also finally getting work done with Hack College (One of the baby Rev3 Beta shows) and am in the process of uploading it. Let’s walk through how I did this finally because you might be curious.

I know that they have the series up on http://blip.tv/hackcollege. This is good for starters. I also know that youtube-dl supports Blip.tv. Excellent. After a little trial and error, I settled on this command:

youtube-dl -c -i -t http://blip.tv/hackcollege

This downloads all the videos from the hackcollege account, puts titles in the file names, continues incomplete downloads, and skips errors. I only added that last part because one video gave me an error (I eventually just downloaded it manually). Then, I forgot I wanted descriptions for each video, so I ran this:

youtube-dl -i -c -t –write-info-json http://blip.tv/hackcollege

See how easy that was? JSON descriptions in just a few seconds. Okay. A few people still know a little trick for getting RSS from Blip.tv: just add “/rss” after the account url. You can go ahead and load it yourself at http://blip.tv/hackcollege/rss. Now, if you’ve been downloading along, you might notice that youtube-dl only snags the .m4v files and there are these beautiful giant .mov files in the RSS feed. Wouldn’t it be nice to grab these? Unfortunately, you can’t fine-tune youtube-dl for these just yet (maybe if I hunt around in the source code I can set this up, but I honestly didn’t think of that until right now). How do we get these files? I came up with this one-liner:

curl http://blip.tv/hackcollege/rss | grep -o ‘http.*mov’ | sort | uniq > out.txt

Pretty self explanatory. Grab the RSS, filter for .mov links, and sort it to get rid of duplicates. Then, save it to a text file. You might notice that the RSS feed doesn’t contain links for every video. There really isn’t anything we can do about that, but it does appear that the .mov files were a semi-recent addition to these videos and earlier videos most likely don’t have the option. I’d say it’s a pretty safe bet that we’ve got them. Next, we will want to download those links. You can probably pipe that one-liner into wget, but I wasn’t ready to download when I wrote it, so I saved the text for later. Here’s how I eventually used it with wget:

wget –user-agent=”iTunes/10.6.1″ -c -i out.txt

It appears that Blip.tv white-lists your user-agent, so it knows to not allow wget, or probably a slew of other bots and/or browsers. You can find this out if you can download a file in your browser but running something automated on the site results in redirects and file fragments. There’s always a workaround. If you trick it into thinking you are iTunes, you not only get access to the files, but also get them faster than if you just manually downloaded from your browser (throttle-free!). Very nice. So after feeding the text file into wget, the .mov files download rather quickly. You don’t get the nice fancy file names like with youtube-dl, but you do get the files.

So there. Only took an hour or two to work everything out.

 

Well, as you can see, I’m finishing up a few parts of SaveRev3. Unfortunately, there is still work to be done and the list keeps on growing.

Here’s to another summer of hard drives and bandwidth. It’s going to be a hot one.

Tags: , , , ,

Other People’s Stuff


If you collect as many things as I do, you end up with some stuff you’re not supposed to have. In this case, I’m not referring to stuff that is illegal or stuff that is unreleased. I’m talking about other people’s stuff. Personal stuff.

In a broad sense, the whole used market is a little bit bizarre when you look at it abstractly. As we live in a disposable culture, anything someone might buy has its own story. You don’t know how many hours were put into that dusty SNES with a five dollar sticker on it. How it contributed to sibling rivalries or became an item to bond over with the girl next door. We’ll usually never really know the extent of these stories.

Sometimes we accidentally inherit the stories.

In Philadelphia, there are many little second hand shops that line the grid-like streets. Tucked into corners, away from the tourist traps, these stores don’t feel like your normal thrift shops. These places lean a little more towards collectibles: antique books, crates of records, obscure (but not rare) VHS tapes- you get the idea. Some of these shops also sell photographs, but not of any famous attractions or curios of the city. They’re family photographs. Weird pictures of people posing outside their houses, sitting with their pets, or just acting goofy. Private pictures. Who would have ever thought they’d end up at a store somewhere?

So every one in a while, I buy a few. I have them scattered along the edges of my bedroom mirror. Who are these people? I’ll never know.

IMG_0809

IMG_0811

IMG_0814

This concept doesn’t simply apply to pictures. From one auction, I got a lot of around thirty 7-inch tape reels. While a lot of them were simply recordings of the radio that could be played back for hours, some of them appeared to be homemade recordings. One I remember in particular appeared to be a recording of some sort of part, complete with almost unintelligible voices and faint background music. Something never meant to get out this far. A memory I own that isn’t mine.

Home movies are another area. On occasion, I’ve purchased VHS camcorders with tapes still inside. Rarely though will I find something captivating. Usually, there will be a short video of a newborn baby or the typical “I’m testing out the camera” tape where people pan around their living rooms.

Occasionally though, I’ll find something more interesting.

One flea market I frequent in Delaware usually has a lot of vendors from house clean-outs. They’re easy to spot: Giant rented truck with several dozen cardboard boxes packed full of everything imaginable. No rhyme or reason here: folded up clothes, kitchen appliances, weathered books, etc. Almost as though a family was packed away into boxes to be sold for five dollars a pop. Anyway, while most of these boxes are filled with junk, I’ve found my fair share of interesting objects from them. At one point, I came across a box of hand-labeled VHS tapes. I didn’t know what exactly they were, so I took them home and played them. I found myself with what appears to be recordings of an amateur band (or several) from the mid 1990′s.

So what did I do with them? After setting them to the side for some time, I decided it was best to transfer them and let people see them. Currently, I have one already online with more on the way. Here’s a link to go watch it. Who are these people? What’s the name of the band? I have no idea. The band might be named “Triple X” but nothing seems to enforce that. Maybe I’ll run across something as I keep going, or someone will stumble upon this video and recognize it. There’s a lot more footage to look through.

So this gives you something to think about. Be mindful of those little personal artifacts that you keep around. Those memories frozen in time. Who knows where they will end up one day.

And who knows if some 20-something punk will eventually put them on YouTube.

Tags: , , , , , ,

Hacker Zines


I have many projects. Too many, one might argue. Either way, they exist and I enjoy doing them.

For a long time, I’ve had something of a collection of magazines that I usually refrain from talking about simply because it doesn’t come up a lot in normal conversation. A few months ago when it was announced that Nintendo Power was halting production, someone told me that the cover of the last issue was a throwback to the very first issue from 1988. I was asked if I had the first issue (people tend to wonder just how much old stuff I have) and I do. Here’s a picture of it.

Nintendo Power #1

Nintendo Power #1

What you don’t see in this picture is the rest of my magazines. I have a lot. Hundreds. Most of them are video game magazines from the 1990′s and I’ve been accumulating them for over ten years. That isn’t to say that these sum up my entire collection. I have an almost complete run of 2600, six or seven years of Wired, a few dozen issues of MAD Magazine from the 1970′s bundled away, 10 or so issues of High Times from the late 1970′s and early 1980′s, and a few years of some more modern things. Besides those, I have a few other random magazines here and there and most likely some I’ve forgotten.

Magazine Shelf

Magazine Shelf

While I have a few current subscriptions, I’ve recently re-opened my magazine obsession. Why now? I don’t really know, but it was bound to happen. Every once in a while you get one of those “I should really do that, wouldn’t that be great?” ideas and they really start to stack up. One or two of those ideas end up toppling off the pile sooner or later and you just run with them. This particular idea started with Blacklisted! 411.

If you haven’t heard of Blacklisted! 411, I don’t hold it against you. If you know what 2600: The Hacker Quarterly is, then think of Blacklisted! 411 as a lower budget version of that. If you don’t know what 2600 is, it’s the most popular and longest running independent print hacker magazine. Blacklisted! has something of an interesting past. There are a lot of politics involving the magazine that are still something of a hot issue even for people today. There’s no doubt that it left a sour taste in the mouths of many. To briefly go through things, the zine started as a cheap black and white publication in the early 90′s. Initially monthly, the magazine switched to a quarterly release schedule to allow for more articles per issue (mirroring 2600 in this regard). Many criticized the quality of the articles and the publication in general, but it had a loyal group of fans and writers. In the mid 1990′s, the magazine up and disappeared (angering many) and reappeared in the early 2000′s. Throughout the life of Blacklisted!, a lot of people claim to have been treated unfairly by it and promised compensation for their articles which they never received  I wasn’t there, and I don’t know all the details for sure. Defending nor attacking the magazine are not my goals either way.

Issues of 2600

Issues of 2600

For as long as I had known about 2600, I had also known about Blacklisted!. While I could easily get back issues of 2600 through their website, Blacklisted! was far more elusive as it went out of print. I was less likely to come across old issues out at book sales or flea markets when compared to something more popular like Wired. So, I forgot about it for a while and chalked it up to a boat that I had missed.

Fast forward to now. I’ve decided to take it upon myself to start gobbling up every issue of Blacklisted! 411 ever produced. Normally when you see someone take on a pie-in-the-sky task like collecting all of something from scratch you dismiss them with an “oh, that’s nice” and pat them on the head while taking bets on how quickly they tire of the project and go home. I already know it’s not something that will happen overnight, and will probably take years if I’m ever able to complete it at all. It’s a bit of a turn-key project either way, so it’s not much of a hassle. Initially,  I set up some aggregation online to see if any issues go up for sale, at most I might dig through a few more bins at the punk rock flea market. It’s something of a slow burn.

Blacklisted! 411

Blacklisted! 411

As I started doing research on Blacklisted! I came upon a few other hacker or hacker-related magazines that went into print. For example, I discovered Mondo 2000 (and its other incarnations), bOING bOING, THUD, Grey Areas, Binary Revolution,  and more. These were also low-number interdependent physical magazines that lived a short life of usually fewer than 20 issues. So, I expanded my scope. If I can find them for the right price, I’ll snatch these up as well. Are there more out there? Probably (And please, let me know what I missed). I can’t get to everything, but I have a pretty good idea of what print zines we had just by asking around.

IMG_0735

Mondo 2000

IMG_0737

Gray Areas

IMG_0738

bOING bOING

IMG_0739

THUD & Binary Revolution

You may raise the issue of me going after physical magazines exclusively. Where’s the love for the electronic zines? While I do have a fondness for ezines, I don’t consider them nearly as endangered a species as the print-only zines. While an electronic zine may have been copied hundreds of thousands of times with little effort, when a physical magazine goes out of print it can only slip further into obscurity. Some copies get mistreated and trashed, while others are packed away and forgotten. These are the ones I want to save. Right now at least.

So the next logical question is what am I doing with all of these magazines? While I admit that I do get a nice warm, fuzzy feeling from physical magazines, I have bigger plans than simple self-satisfaction. Scanning is the name of the game. I’m currently in the process of scanning in all these old issues I’ve already found, compiling each issue into a single document, and uploading the documents online to share with everyone. Through this whole scanning process, I’ve already learned a lot. Enough to write something on it actually, but it would fare better as its own article. My scanning workflow works well enough to actually yield results, which you can check out here and here. If you want to check out my overall progress on how I’m doing with all the zines I hope to find, you can visit this page. You might notice that in some cases, I’ve found magazines already scanned by people. These are few and far between, but save me a little work considering they are usually of good quality.

While my scanner might be slow and I might be busy, I’m happy to say that the wheels are in motion. Things would probably move a little faster if I had a more portable scanner, but for the time being I’m keeping things slow and steady. That all said, if you have some of these magazines and feel like donating to the cause, I’ll serve as a home for your wayward magazines (and I’m probably a decent alternative to the trash if anything). If you feel like scanning, you can contribute that way as well. The Anarchivism wiki linked above is editable if you create an account.

So as I’m picking up older magazines, I’m also starting to focus on newer ones. Consider something like Bitcoin Magazine. An independent publication about a decentralized digital currency? Who knows how much longer this will stick around. It’s important to apply a little foresight for things like this. Otherwise, who knows what you’ll be able to get your hands on down the line. Luckily, many current publications have bridged the digital divide and offer both physical and electronic copies. Other magazines are now entirely based online. Still, there are those holdouts that are only available on paper. These are what I’m after. These are what I want to save before time runs out.

 

Tags: , , , , , , , , ,