Sunday, 9 December 2012

Amarok 2.7 Beta To Be Released Soon, Try Statistics Synchronization!

Heya, Amarok 2.7 Beta is slated to be released in any day now, one of the features it brings is Statistics Synchronization, a result of my GSoC project. Here's a short guide to get the best from it.

Unattended Synchronization

Getting started is utterly simple, just answer Yes to the above question that pops up when you connect a capable media player (currently just iPods) and you're done! Amarok will work hard in the background from that point to make sure your ratings, first/last played times, play counts and labels are synchronized between all participating collections (Local Collection is included by default) and won't disturb you unless there's a conflict. If you have Last.fm plugin configured, tracks recently played on your iDevice will be automatically scrobbled to Last.fm as part of the synchronization.

Conflicts & Synchronize Statistics Dialog

When you for example change a song rating simultaneously in Amarok and on your iPod, Amarok doesn't know what rating it should use. In this case, the last step of the synchronization is made interactive: a dialog with synchronization preview shows up. On top, you can see a few tabs:
  • Matched Tracks: the essential tab where the tracks that Amarok found to be in more than one of your Collections are grouped together. You can tell which tracks and fields are going to be updated using the background: light green above means a new/updated field and light red means an old/overwritten one. In case of rating or label conflict you can resolve it using the check boxes. Conflicts can also be resolved in batch using the buttons below the list, which affect all applicable tracks, not just the shown ones. You don't need to resolve all conflicts, the particular field of the particular tracks won't be synchronized.
  • Unique Tracks: a list of tracks found just in one collection. You can use this as a some kind of a difference view of your collections. This tab supports track dragging.
  • Excluded Tracks: sometimes a track matching is ambiguous. This can happen for example when you have 2 tracks with identical meta-data in your collection. Such tracks are excluded from synchronization in order not to cause disorder.

Manual Syncing & Last.fm

At any time, you can start the synchronization manually from Amarok Tools menu using the Synchronize Statistics... entry. You can override what collections and fields to synchronize in this case. If you have Last.fm plugin enabled and configured, you can also choose to synchronize your play count and labels with Last.fm; unattended synchronization with Last.fm is not possible, because that would put too much stress on their servers. Also please note that track matching when Last.fm is selected can be very time-consuming (one hour is not unusual), but you can use Amarok freely while this happens.
Synchronizing with Last.fm is a bit tricky, because by default it auto-corrects some common misspellings during scrobbling. So it can happen that even if you scrobble many track plays, it doesn't show up in Matched Tracks because Last.fm knows it under a slightly different name. Fortunately you can opt-out from auto-correction on Last.fm site and the change is applied also to your past scrobbles, both options have some downsides though:
  1. auto-correct off: your tracks are matched properly, but if you change your tags after some time (because of an actual typo for example), your play-count on Last.fm site will be split into 2 entries.
  2. auto-correct on: can track play counts even if you correct the tags over time, but you have to use Last.fm's preferred spelling or your track's won't match.

Configuration

You can configure all aspects of statistics synchronization in a new Amarok Configuration tab named Metadata. Apart from configuring which collections should be synchronized automatically you can decide what fields you want to synchronize and edit a list of labels excluded from synchronization (useful if don't what some of your Amarok labels to appear on your Last.fm profile).

Technical Remarks

Curious what the internal algorithms and principles of the synchronization are? Here you go:
  • Tracks are matched using their meta-data case insensitively. The framework itself and both iPod and Local Amarok collections can match using: title, artist, album, composer, year, track number, disc number. Last.fm can only match using title, artist and album. Common subset of matching fields of selected collections is used. That's why you can get tracks excluded from synchronization due to ambiguity when you include Last.fm in syncing.
  • Local Collection can synchronize everything.
  • iPod Collection can synchronize everything except labels, although and iPod won't update first played time by itself.
  • Last.fm can read play count, labels and rating (see below) and it can write labels and rating; play count is updated by means of scrobbling. In theory, first/last played time should be available too, but their API doesn't export it currently in a efficient way.
  • Last.fm rating is implemented using special tags like 4 of 10 stars. You can opt-out from using these, then Amarok will ignore this kind of tags.
  • Conservative reduction functions are used for synchronization of individual fields, for example minimum-of is employed for first played time and maximum-of for last played time. Play count uses maximum-of + special case for iPods that can report recent play count so that you don't miss anything even if you play a track simultaneously on an iPod and Amarok.

Tips

  • Use Last.fm synchronization as a back-up of your precious Amarok data!
  • Use Last.fm synchronization as a tool to synchronize your listening data from multiple places!
  • New! Use inter-collection synchronization and the new Nepomuk Collection to keep your Amarok ratings in sync with your Nepomuk (i.e. shown in Dolphin) ones. Nepomuk collection should be able to handle more fields in future.

Test it!

I've worked on this and tested it myself for a rather long time, but there will be for sure some bugs that got through. Please test this in Amarok 2.7 Beta or current git and report any bugs you find so that I have a chance to fix them for Amarok 2.7 final. Thanks and have fun!

12 comments:

  1. Finally. I love you, guys.

    ReplyDelete
  2. Xavier Corredor Llano9 December 2012 at 20:37

    wuau, the statistics synchronization with last.fm is so useful, I love it, thanks so much!

    ReplyDelete
  3. Amazing work! Thanks Matěj!

    ReplyDelete
  4. Congrats Matěj, awesome work, awesome improvment !

    ReplyDelete
  5. awesome feature, thanks Matěj!

    ReplyDelete
  6. Pořád mi v Amaroku chybí spotify nebo něco na ten způsob....tak snad příště.

    ReplyDelete
  7. Spotify plugin se také udělal během tohoto léta jako GSoC projekt, ovšem kvůli zatím nevyřešené licenční situaci (oprávnení k distribuci jejich uzavřené knihovny) nemůže být zahrnuto v Amaroku 2.7. Pokud ti nevadí kompilovat, pingni chlápka Stecchino na #amarok IRC.

    ReplyDelete
  8. All nice, but till 2.6.x i still can't sync my Xperia S Android phone which, excuse me sucks. You should focus on MTP devices too. Even if Amarok discovers it, it shows all tracks like "uknown artist" or, when it does work right it takes AGES to finish sycing. Hope you fix that soon, too.

    ReplyDelete
  9. Would you buy me a Samsung S III Mini to have something to develop it against? :-) Or even better, would you invest your time demanting nothing in return (like we do) and implement it?

    ReplyDelete
  10. Hi! I was surprised in a good way when I loaded this web page of your resource. What was the main reason at that moment when you decided to organize a site?

    ReplyDelete
  11. Sweet! At last, last.fm sync! My prayers were heard.

    ReplyDelete