These are freely available datasets from here and around the world. Some are accessible via API's, which are an automated way to mine data into software like Google Refine.

Data.gov.au - [2] Is hosted by the Australian Government. At present it contains over 1000 datasets, including a few datasets on Music. Contains a bunch of interesting apps and links to resources like GIS and Datawrangler.

Datacatalogs.org [3] - Is a portal to open Datasets from around the world. API compatible.

Eurostat [4] - Business and demographic statistics from the Eurozone. Also contains specialist datasets like culture.

World Bank Data Catalog [5] - A wealth of datasets from all parts of the globe. Contains World Development Indicators, World Bank Finances, Health, Education Stats and specialist sets like the Knowledge Economy. Largely API compatible.