Bulk Downloading Option?

Moderators: kcleung, Wiki Admins

Danedaworld
Posts: 1
Joined: Sun Aug 29, 2010 4:58 pm
notabot: 42
notabot2: Human

Bulk Downloading Option?

Post by Danedaworld »

I was wondering if there was anyway to bulk download music with just 1 click. I'm having such a hard time getting all of the music on lets say Chopin's page that its taking me a few hours to get only just 2/3's. If the people could just find a way to bulk download like all of the music from 1 composer or a group of composers, that would be great! Especially if there was a way to bulk download/P2P all of the sheet music with a few clicks!
reinhold
forum adept
Posts: 54
Joined: Mon Jun 02, 2008 6:00 am
notabot: YES
notabot2: Bot
Location: Vienna, Austria
Contact:

Re: Bulk Downloading Option?

Post by reinhold »

Why would you need ALL scores of a composer? If you want to do some real work (i.e. perform something, or look for a fitting piece for some occasion), I can't imagine a situation where you would need all scores of a composer.
On the other hand, if you simply want a collection of as many scores on your harddrive (or worse, on the internet on a different site), I don't think IMSLP should help you too much with that (in particular, since that takes large amounts of bandwidth, which is then missing for "normal" users)...
Melodia
active poster
Posts: 442
Joined: Sat Oct 20, 2007 12:30 pm

Re: Bulk Downloading Option?

Post by Melodia »

I at one point had DLed all of the scores of quite a number of composers, as part of my personal "completion projects". Though I scrapped the idea...

Still, while it would be able to do it all in one shot, one option might be to use a browser extension such as DownloadThemAll, and all you'd have to do is right click once on each page (though of course, that gets all editions, arrangements, etc).
From what I can tell, there's no real correlation to where the order the files are stored relative to who they are by, so there'd be no plausible way to get everything in one shot.
vinteuil
Groundskeeper
Posts: 1445
Joined: Sun Oct 05, 2008 3:01 pm
notabot: YES
notabot2: Bot
Location: U.S.A.
Contact:

Re: Bulk Downloading Option?

Post by vinteuil »

Bh2000 which has a good number of IMSLP's "core scores" (good stuff!) has a bittorrent option which works nicely.
Formerly known as "perlnerd666"
BKhon

Re: Bulk Downloading Option?

Post by BKhon »

Actually, I don't think the ability to Bulk download is a terrible idea. I agree with reinhold on most of his points, but the option of downloading, say, the complete symphonies of Beethoven, or all the parts to some piece, then it should be fine (except that it would slow down IMSLP). And also, maybe Danedaworld wants to start a personal music library and bind all the scores, or something.
KGill
Copyright Reviewer
Posts: 1295
Joined: Thu Apr 09, 2009 10:16 pm
notabot: 42
notabot2: Human

Re: Bulk Downloading Option?

Post by KGill »

It's true that there might be some sort of consequence for our servers - site ripping is always a problem and there's a script in place to stop it (this could come dangerously close). Not sure what Feldmahler's thoughts are, though...
arkiados
Posts: 10
Joined: Wed Feb 23, 2011 3:46 am
notabot: 42
notabot2: Bot

Re: Bulk Downloading Option?

Post by arkiados »

I would like this feature as well.

Publishing the entire score database quarterly via bittorrent or though open mirrors would help to preserve the music. Many "open-source" projects already do this. Publishing through bittorrent wouldn't put unnecessary strain on bandwidth or server load. I would be glad to contribute my time to such a project if needed.

It is correct that I could never use all of the scores but I would hate for something to happen to IMSLP (copyright suit, natural disaster, economic meltdown, etc) and all the hard work be lost. I would still use the website to browse scores and could then look them up in my local cache. Category Walker is amazing!

Thoughts?
pml
Copyright Reviewer
Posts: 1219
Joined: Fri Mar 16, 2007 3:42 am
notabot: 42
notabot2: Human
Location: Melbourne, Australia
Contact:

Re: Bulk Downloading Option?

Post by pml »

The entire IMSLP collection includes a significant proportion of scores that are published under a variety of copyrights and licences, which cannot simply be treated as a single monolithic download – to do so would likely infringe multiple copyrights somewhere. However, I would see no difficulty in making torrents of several of the main divisions (according to the copyright tagging scheme), for example, the scores that are pure “public domain” in all three domains we consider (Canada/US/Europe).

Cheers, Philip
--
PML (talk)
arkiados
Posts: 10
Joined: Wed Feb 23, 2011 3:46 am
notabot: 42
notabot2: Bot

Re: Bulk Downloading Option?

Post by arkiados »

Philip,
That makes perfect sense. Scores in the public domain would be of the most importance in my opinion. How do we proceed with this idea?

Thanks, David
pml
Copyright Reviewer
Posts: 1219
Joined: Fri Mar 16, 2007 3:42 am
notabot: 42
notabot2: Human
Location: Melbourne, Australia
Contact:

Re: Bulk Downloading Option?

Post by pml »

The statistics from the copyright tagging system indicate that 66,237 scores are in the public domain in Canada without any qualification*: these vary slightly from the numbers for the US (64,380) and the European Union (53,718). I am not aware of a calculation of the intersection of these three categories, but I am guessing that some files counted in the US figure are not in either of the Canadian or EU totals; thus the maximum size of the main torrent you would want implemented is in the vicinity of 52,500 files – that is, slighly less than the EU sub-total.

It’s not up to me to suggest how this could be implemented, but each of the files tagged with the V/V/V copyright tag could have the matching filename added to a list to allow compilation of this part of the collection into a torrent; however, many of the files are badly named in terms of properly identifying their contents. Many contributors do an excellent job of uploading files with helpful descriptions of the score; some are not helpful at all. For example, many of the scores uploaded from Sibley seem to have numeric filenames with no identification of the work, the composer, or the type of the score. It may be possible to include a text file that identifies the page that hosts a particular work, e.g. imslp # , filename , page title

#10638 , http://imslp.org/wiki/File:MGFischerOrgan.pdf , Selected Organ Pieces (Fischer, Michael Gotthard)
#68439 , http://imslp.org/wiki/File:PMLP138150-e ... _score.pdf , Sonata, Op.7 (Eichborn, Hermann Ludwig)
(2 random page examples)

Regards, Philip

* There are comparably small numbers of files that are in the public domain owing to the interpretation of copyright laws regarding scientific publications (i.e. urtext editions): Canada, 3,311; the U.S., 3,298; and the E.U., 5,181. These are counted separately of the files above.
--
PML (talk)
arkiados
Posts: 10
Joined: Wed Feb 23, 2011 3:46 am
notabot: 42
notabot2: Bot

Re: Bulk Downloading Option?

Post by arkiados »

A simple text file would work well as an index.

Scores that are without reasonable doubt in the public domain would be a safe bet. Meaning those 52,500 files you mentioned.

Any guess as to what the total file size would be?
arkiados
Posts: 10
Joined: Wed Feb 23, 2011 3:46 am
notabot: 42
notabot2: Bot

Re: Bulk Downloading Option?

Post by arkiados »

Also, I would be glad to offer my assistance with the site. I have a strong systems administration background. I won't be much help with regards to the actual music though. I am just now seriously pursuing the piano. Send me a PM if you're interested and I can send my credentials.
zooker
Posts: 4
Joined: Tue Feb 22, 2011 3:20 pm
notabot: 42
notabot2: Human

Re: Bulk Downloading Option?

Post by zooker »

I might suggest a different tack on this problem:


I would prefer an archive of this entire site, without the forums.


The reason for this is two fold:
1. I can traverse this archive of the website in my browser. Just as I do with the live version.
2. My main purpose in creating the archive is to perpetuate the generous work and time all of the users of this site gave in uploading files - and prevent the site being incapacitated, either via a lawsuit, or the owners/board/what-have-you going profit.

An archive of the *entire* site would ensure that "going profit/subscription" will never happen.
And an archive of the *entire* site would ensure that a lawsuit would become impotent (and therefore diminishing the chances of it occurring as well).

If your site is setup via a DB, please produce a torrent of the entire DB.
If your site is setup via static files on disc, please zip up the entire disk.
If your site is a combination of the two, please zip and include the DB.


If you wish to divide the site up via "zoning" copyright stuff, please do.



Does this make sense? I can rip your entire site instead if you wish not to take appropriate steps to safeguard the future "freedom" of this site.
pml
Copyright Reviewer
Posts: 1219
Joined: Fri Mar 16, 2007 3:42 am
notabot: 42
notabot2: Human
Location: Melbourne, Australia
Contact:

Re: Bulk Downloading Option?

Post by pml »

Zooker, my memory might be at variance with the facts, but were you the guy who posted here recently about the possibility of a Korean sitemirror?

Also bear in mind that in part, the IMSLP collection is geographically divided – some of the files being accessible by redirection from the wiki to independent servers in the European Union or the United States. To my knowledge there is no country where all files accessible from IMSLP can be downloaded legally, except for those which are completely outside any recognisable legal framework (e.g. Somalia or Afghanistan). In other words, to quote you,
If you wish to divide the site up via "zoning" copyright stuff, please do.
As my previous post in this thread made clear, that is a sine qua non – it already is.

Cheers PML
--
PML (talk)
arkiados
Posts: 10
Joined: Wed Feb 23, 2011 3:46 am
notabot: 42
notabot2: Bot

Re: Bulk Downloading Option?

Post by arkiados »

Zooker,
It seems that we want the same thing and those in charge aren't completely opposed to the idea. Your message comes across with a bit of hostility. If we are going to get this done we need to do it right and in accordance with the law. This process is going to take some time and would probably be faster if someone "close" to the servers got it done. These admins would be able to schedule the data transfers at an off-peak time that would not affect standard users. We would also need a schedule and definitely a way to automate the procedure.

I do agree that it may be a good idea to have a browsable "local" copy of the site. However, we still don't have an answer to the size of the score database. Adding audio and video into the mix will make the file size EXPONENTIALLY larger.

Thanks
Post Reply