These are freely available datasets from here and around the world. Some are accessible via API's, which are an automated way to mine data into software like Google Refine.
Data.gov.au - Is hosted by the Australian Government. At present it contains over 1000 datasets, including a few datasets on Music. Contains a bunch of interesting apps and links to resources like GIS and Datawrangler.
Datacatalogs.org - Is a portal to open Datasets from around the world. API compatible.
Eurostat - Business and demographic statistics from the Eurozone. Also contains specialist datasets like culture.
World Bank Data Catalog - A wealth of datasets from all parts of the globe. Contains World Development Indicators, World Bank Finances, Health, Education Stats and specialist sets like the Knowledge Economy. Largely API compatible.