So…there is this whole thing about Soundcloud shutting down. Soundcloud says no. Tons of websites say yes. We have already led you where to go now that Soundcloud may have a terrible fate. This post is how to backup your files from Soundcloud just in case you don’t know how to backup your files and now are scrambling to get your archive before it disappears. I don’t blame you. This is where I come into help. I will teach you multiple ways to download your own music off of Soundcloud in order to prevent any file loss.
Before we start, I want to note the following statement for legal reasons.
The following information below is solely for the purpose of backing up your own files and nothing more. Getting any other music or doing a full grab of the site is considered mass copyright infringement. WE DO NOT CONDONE SAID ACTIONS. DON’T SUE US FOR SHARING INFORMATION.
All of the music files that are currently streaming on Soundcloud are at 128k MP3 streams. This means, they aren’t the best quality, but when you don’t have your music anymore, you just have to accept this.
Internet Download Managers
Internet Download Managers are programs where they scour pages you tell it to scour and find all the files you can potentially download. There are many of them out there. They are great when you use programs and want to download a ton of files, but don’t want to boggle down your internet as well as your computer. There are many of them available. Wikipedia has a great table that explains what they can and cannot do. You can find that link here. The one I currently use is JDownloader 2. Another good one is the Internet Download Manager. Find one that works on your operating system and go to down. Essentially, all you have to do is copy your link, put it into the program, and it will list all your music files available to download.
Note: When you use JDownloader (for example based of what I have experimented with) on your Soundcloud profile. It will also pick up all the music that you have reposted as that is shown on your profile. ONLY DOWNLOAD YOUR FILES.
Github Projects
Many coders have been uploading their programs to Github for people to use to backup their files. Currently from what we found was the Soundcloud Music Downloader and Soundscape. Both are recommended by the people over at /r/DataHoarder. Both of these programs are contributed by fantastic coders to make sure you can get your files. Also available is the YouTube-dl which may not have Sound or Soundcloud in the title, but lists Soundcloud under their supported sites available to use.
Soundcloud Ripping Websites
Don’t even bother with these sites. Don’t rely on websites because this is how we ended up in this scenario to rip all our old music off of Soundcloud because we didn’t backup properly. You are limited to their server speed, their connection to the internet. It is a waste of time. There are many options available regardless of the operating system you’re using. Worse case scenario, use a virtual machine (because if you can’t get a program to run on your operating system, you sure as hell know what virtual machines because you clearly don’t have Mac or PC or Linux.)
Theoretical Soundcloud Backup Project
Like Mythbusters, I wanted to look into a theoretical project where I could potentially backup all of Soundcloud because I don’t want to lose it. I do want to note, this is theoretical because this is mass copyright infringement and also you’ll learn real quick why I can’t afford this. According to the #ArchiveTeam, as of April 2017 the 128k MP3 streams of all public tracks on SoundCloud total 855TB. For estimate purposes, the total amount by now is anywhere between 900TB and 2PB (or 2,000TB). That is a lot of goddamn storage. This is where people would push the limits of what unlimted storage means with Amazon and Google. Although, let’s go further.
If there is anywhere between 900TB and 2PB, that means we need to be able to handle them all. The highest amount hard drives go to now (for purchase) is 10TB. You have a few different hard drive choices you can go with. I’ll note them below.
- Western Digital Red
- Western Digital Red Pro (coming soon)
- Western Digital Gold
- Seagate BarraCuda Pro
- Seagate IronWolf
- Seagate Enterprise Capacity
- Seagate IronWolf Pro
- Hitachi Ultrastar He10
These are stupidly expensive, but for good reason. They aren’t your normal consumer hard drives. They are meant for all types of servers.
Because this is all theoretical, I am going to go with the cheapest drive even though theoretically, I’d go with the most expensive if I were holding everyone else’s music because of potential drive failure. The theoretical cheapest isn’t a 10TB hard drive, but rather an 8 TB hard drive. It is also a big pain in the ass because you have to shuck the external part of the external hard drive to get to it. Anyway, the cheapest per GB is Western Digital easystore® 8TB External USB 3.0 Hard Drive available from Best Buy. These are always on sale at Best Buy for $199 rather than the actual price of $299. This makes the price per GB $0.025 vs. the cheapest 10TB – the Seagate IronWolf which comes in at $0.034 per GB.
Side note: Upon further investigation, while the Western Digital easystore® 8TB External USB 3.0 Hard Drive is still the cheapest, the 8TB Seagate Archive, 8TB Seagate BarraCuda Pro, 8TB Seagate IronWolf, 8TB Western Digital Red and the 8TB Hitachi Deckstar NAS were all cheaper per GB than the Seagate IronWolf 10TB. Although, the Western Digital easystore® 8TB External USB 3.0 Hard Drive has an 8TB Western Digital Red inside. So it is technically $64 cheaper than buying it without the case.
So you’re buying the Western Digital easystore® 8TB External USB 3.0 Hard Drive. Perfect. You cannot run these hard drives by themselves. You need to put this many hard drives in a RAID array in case a hard drive or two fails and we don’t want to lose all the data. That means we need more hard drive space than the total amount of music for redundancy.
Let’s say that there is more music than we anticipated and we’ll leave room. We’ll say there is a total of 2.5PB of music. That means with a raid array of RAID 60 because it is my understanding that it is the best type of RAID Array for this many hard drives. Although, you’ll see how much more in-depth this goes. To reach 2.5PB with a little bit of extra room because we can’t fill this up to the brim as this needs to run. According to a RAID calculator, we would need 350 8TB hard drives. Being that Best Buy only allows us to buy 3 hard drives at a time, you will need to make 116 orders of 3 hard drives for a total (including tax only as shipping or store pickup is free) of $74,381.52. Don’t forget the 117th order for the 2 remaining drives for $427.48. This brings the total drive cost to $74,809.00. This doesn’t even consider if any of the drives are bad and the warranty because we’re shucking them.
Now that we bought hard drives, we need to hold them all and run them. You could potentially build your own solutions, but being that I am not as familiar with this process, the Storinator XL60 holds 60 drives per server and is already premade. Since we’re sticking with the cheap route, their cheapest version for one server $6930.00. Being we need 6 of them (even though it isn’t recommended to have THIS MANY hard drives in one server for protection purposes, we’re channeling our inner Linus Tech Tips), this will come out to $41,580.
None of this includes the network hardware to link six 60 hard drive servers together nor does this include the electric bill to create this huge homelab or any of the other hardware you need. Plus, it isn’t recommended to do it this cheap especially with this much data. Needless to say, you’re already $116,389.00 into this project and it isn’t even worth it.
We aren’t done. Redditor, appropriateinside, did some math based on the approximation that Soundcloud is closing in 50 days. Techcrunch updated their article to say 80 days, but this was under the assumption they were going to close in 50 days and have ~1.9PB of music.
The only difference is that I will edit what he said based on the updated timeline. This would be 2.22Gb/s for all 80 days, that’s one (estimated) song every 18.07ms.
This is based upon the fact that you have steady electricity and internet for the next 80 days along with all the servers and such setup. Worth it? Not at all.
We have approximately 80 days until the alleged shutdown of Soundcloud. That means that there is going to be failure leading up to the inevitable end. They have to consolidate offices and they let go a ton of staff. It is going to happen. Just watch. Rather be on the forefront of backing up your files before it is too late. Also, don’t steal but who am I to stop you from living your life and doing whatever you want.