Overview
This page provides instructions for creating a dump of your last.fm listening history and import it into libre.fm. These instructions are geared towards Debian and Ubuntu GNU/Linux users. Feel free to add instructions for running these scripts on your operating system of choice. There are two scripts involved with this process. The first, lastscrape.py, is a python program for pulling listened tracks from last.fm. The second, import.py, is a python program for pushing listened tracks.
Important Notice: if your last.fm profile is set to "private" the lastscrape.py script won't work!!
Scraper GUI
An easy-to-use program for migrating your listening history may be at http://cs.joensuu.fi/~pviktor/lastscrape/. It provides an interface for all the steps below (scraping, removing duplicates, and uploading; obtaining the prerequisites may also be easier, especially for Windows users).
Obtaining lastscrape.py and import.py
These two scripts are located in the libre.fm subversion repository.
import.py also depends on gobble.py, which you should put in the same directory as it:
Installing Dependencies
Install the dependencies for lastscrape.py
$ sudo apt-get install python-beautifulsoup
Beautiful Soup 3.1.0 is known to give errors. http://www.crummy.com/software/BeautifulSoup/3.1-problems.html Problems have also been reported with version "3.0.7-1" from Ubuntu repositories. Use version 3.0.7a instead. You may use 3.0.7a by placing a copy of the file locally:
$ cd /path/to/lastscrape.py $ wget http://www.crummy.com/software/BeautifulSoup/download/3.x/BeautifulSoup-3.0.7a.py -O BeautifulSoup.py $ ./lastscrape.py
Running lastscrape.py
$ ./lastscrape.py INSERT_LASTFM_USERNAME_HERE lastfm_dump.txt
You might need to run
$ chmod +x lastscrape.py
first, to get it executable.
Running import.py
This is important, before importing, you should check your dump file for duplicate entries otherwise the import might crash half way which is not a pleasant experience. If your last.fm account was created before 2007, you will probably have duplicates entries due to last.fm bugs. To remove the duplicates lines in the dump file, you can open the file with vi and do:
:%s/^\(.*\)\n\1$/\1/
Or you can just type from console (under Linux/Unix with awk installed):
$ awk '!x[$0]++' lastfm_dump.txt > lastfm_dump_no_duplicates.txt
To check whether the duplicates have been removed, do:
$ cat lastfm_dump_no_duplicates.txt | wc -l; cat lastfm_dump.txt | sort -u | wc -l
And check that the numbers are the same. There are many other ways to remove duplicates, just check online http://www.google.com/search?q=remove+duplicates+lines+from+file.
Remove the first line ("Artist Track Timestamp") of the dump file with an editor.
$ ./import.py INSERT_LIBREFM_USERNAME_HERE lastfm_dump.txt
The script will ask you for your password and then submit tracks in batches of 50.
If your username has spaces, replace them with +. Eg: if your username is "John Doe", then do
$ ./import.py John+Doe lastfm_dump_no_duplicates.txt
If you get gobble errors related with BADTIME just synchronise your system date using ntpdate
# ntpdate pool.ntp.org
If you are trying to submit to a different server, use the -s option. For example:
$ ./import.py -s http://turtle.libre.fm/ John+Doe lastfm_dump_no_duplicates.txt
Other distros
You're invited to add distro-specific instructions here.
Arch Linux
As lastscrape doesn't work with Beautiful Soup 3.1 you'll have to downgrade momentarily to 3.0.7a.
Search for beautiful-soup PKGBUILD on /var/abs/community/devel.
Then change pkgver to 3.0.7a and source to http://www.crummy.com/software/BeautifulSoup/download/3.x/BeautifulSoup-$pkgver.tar.gz. Remove the md5sum line.
Also, you can get the beautiful-soup 3.0.7a from community CVS. Just remember to change the source url.
Run makepkg -i and you're done.
Fedora
In Fedora 10, install Beautiful Soup using yum:
# yum install python-BeautifulSoup
Follow the instructions as shown in Debian/Ubuntu section.
Gentoo Linux
# emerge -av beautifulsoup
Follow the instructions as listed in the general sections above.
openSUSE
In openSUSE (or SuSE), install Beautiful Soup using zypper:
# sudo zypper install python-BeautifulSoup
Follow the instructions as shown in Debian/Ubuntu section.
Mac OS X
Just follow the instructions as listed in the general sections above, downloading BeautifulSoup with the wget command instead of apt-get.
Windows
Get the installer for Python 2.5 (tested) or 2.6 from python.org and install.
Download BeautifulSoup 3.0.7 (version 3.1.0 does not work), unzip and put the file BeautifulSoup.py into subfolder ..\Lib\site-packages of your Python install.
Put the two Scripts into your Python folder, run a command prompt and use the scripts with the same commands as above.
So, if your last.fm user-name is user you would type the following into the command prompt in your python 2.5 directory:
python lastscrape.py user myscrapedscrobbles.lfm
Your scrobbled files will start printing out to the screen, once it's done check for duplicates as described and type in:
python import.py INSERT_LIBREFM_USERNAME_HERE myscrapedscrobbles_checkedForDuplicates.lfm
