From a3bdd4b2171144507004ff5b9d3ec4b582194b29 Mon Sep 17 00:00:00 2001 From: Johan Lundberg Date: Wed, 2 Jul 2025 00:36:58 +0200 Subject: [PATCH] Add Spotify integration with album streaming links MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Implement Spotify Web API integration for album streaming links - Add extract_spotify_urls.py script for automated Spotify URL extraction - Create spotify_urls_mapping.json with sample album mappings (20 albums) - Update album cards to include both Wikipedia and Spotify links - Add Spotify-branded styling with official green color and logo - Implement smart fallback to Spotify search for unmapped albums - Add responsive design for mobile with stacked link layout - Update README with comprehensive feature documentation Features: • Each album now has "Listen on Spotify" link with Spotify icon • Spotify links use official Spotify green branding • Theme-aware styling adapts to dark/light themes • Mobile-optimized layout with vertical link stacking • Production-ready script for extracting all 500 album URLs 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude --- README.md | 62 +++++++++--- extract_spotify_urls.py | 193 ++++++++++++++++++++++++++++++++++++++ script.js | 36 +++++++ spotify_urls_mapping.json | 22 +++++ styles.css | 41 ++++++++ 5 files changed, 340 insertions(+), 14 deletions(-) create mode 100644 extract_spotify_urls.py create mode 100644 spotify_urls_mapping.json diff --git a/README.md b/README.md index 52d3152..1c3e46f 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,29 @@ # 🎵 Rolling Stone's Top 500 Albums -A beautiful, interactive web application showcasing Rolling Stone's greatest albums of all time with visual comparisons between the 2020 and 2023 rankings. +A beautiful, interactive web application showcasing Rolling Stone's greatest albums of all time with visual comparisons between the 2020 and 2023 rankings. Features a comprehensive theme system, Wikipedia integration, and modern responsive design. ![Top 500 Albums](https://img.shields.io/badge/Albums-500-brightgreen) ![Data Complete](https://img.shields.io/badge/Data-100%25%20Complete-success) ![Cover Art](https://img.shields.io/badge/Cover%20Art-500%20Albums-blue) +![Themes](https://img.shields.io/badge/Themes-8-purple) +![Wikipedia](https://img.shields.io/badge/Wikipedia-Integrated-orange) ## ✨ Features +### Core Functionality - **Interactive Album Cards**: Browse all 500 albums with high-quality cover art - **Ranking Comparisons**: See how albums moved between 2020 and 2023 rankings - **Complete Metadata**: Full album information including artist, year, label, and descriptions -- **Responsive Design**: Beautiful layout that works on all devices -- **Search & Filter**: Easy navigation through the extensive collection +- **Wikipedia Integration**: Direct links to Wikipedia pages for each album +- **Search & Filter**: Easy navigation with search, status filters, and sorting options + +### Modern UI/UX +- **8 Beautiful Themes**: Gruvbox (default), Basic Blue, Dark, Gruvbox Dark, Dracula, Nord, Solarized, Arc +- **Theme Persistence**: Your preferred theme is saved automatically +- **Responsive Design**: Optimized layout for desktop, tablet, and mobile devices +- **Clean SVG Icons**: Modern iconography throughout the interface +- **Jump-to-Rank**: Quick navigation to any album by rank +- **Shareable URLs**: Bookmark and share specific albums or filtered views ## 🚀 Live Demo @@ -42,6 +53,7 @@ python -m http.server 8000 - `top_500_albums_2023.csv` - Complete dataset with 100% metadata coverage - `rolling_stone_top_500_albums_2020.csv` - Original 2020 rankings - `wikipedia_top_500_albums.csv` - Wikipedia sourced data for comparison +- `wikipedia_urls_mapping.json` - Accurate Wikipedia URL mappings for all albums ### Assets - `covers/` - 500 high-quality album cover images (rank_XXX_Artist_Album.jpg) @@ -57,16 +69,18 @@ The repository includes various Python utilities for data management: |--------|---------| | `compare_top500_albums.py` | Generate the main comparison dataset | | `merge_descriptions.py` | Merge album descriptions from multiple sources | -| `download_album_covers.py` | Download album artwork from iTunes API | +| `download_all_covers.py` | Download album artwork from iTunes API (500/500 success) | | `add_missing_info.py` | Add metadata for albums missing information | | `fill_missing_from_wikipedia.py` | Research and add Wikipedia-sourced descriptions | +| `extract_wikipedia_urls.py` | Extract accurate Wikipedia URLs for album pages | ## 📈 Data Quality - **500/500 albums** with complete ranking information -- **500/500 albums** with cover art +- **500/500 albums** with cover art (downloaded via iTunes API) - **500/500 albums** with metadata (artist, album, year, label) - **500/500 albums** with descriptions (mix of original Rolling Stone content and researched additions) +- **496/500 albums** with accurate Wikipedia page links (99.2% coverage) ## 🎯 Key Insights @@ -75,7 +89,10 @@ The repository includes various Python utilities for data management: - **New Entries**: Recent albums like Beyoncé's "Renaissance" and Bad Bunny's "Un Verano Sin Ti" - **Genre Diversity**: Increased representation of hip-hop, R&B, and global music -### Statistics +### Statistics +- **New Albums in 2023**: 192 albums (38.4% of the list) +- **Improved Rankings**: 164 albums moved up +- **Dropped Rankings**: 113 albums moved down or were removed - **Most Represented Artist**: The Beatles (multiple albums in top rankings) - **Decades Covered**: 1950s through 2020s - **Genres**: Rock, Hip-Hop, R&B, Soul, Punk, Electronic, Country, Jazz, and more @@ -83,32 +100,49 @@ The repository includes various Python utilities for data management: ## 🔧 Technical Details ### Frontend -- **Vanilla JavaScript** for maximum compatibility +- **Vanilla JavaScript** for maximum compatibility and performance +- **CSS Custom Properties** for dynamic theming system - **CSS Grid & Flexbox** for responsive layouts +- **SVG Icons** for crisp, scalable interface elements +- **LocalStorage API** for theme persistence - **Progressive Enhancement** for accessibility ### Data Processing - **Python 3** scripts for data manipulation - **CSV format** for easy data management -- **iTunes API** integration for cover art +- **iTunes API** integration for cover art (100% success rate) +- **Wikipedia scraping** for accurate page URLs - **Fuzzy string matching** for data correlation +- **JSON mapping files** for efficient lookups ## 📝 Development ### Running Locally 1. Clone the repository -2. Open `index.html` in a web browser -3. For development with live reload, use any local server +2. Serve with a local HTTP server (required for CSV loading): + ```bash + python -m http.server 8000 + # Then visit http://localhost:8000 + ``` +3. For development, any local server will work + +### Theme Development +The application uses CSS custom properties for theming: +- 8 built-in themes with consistent color schemes +- Easy to add new themes by extending the CSS variables +- Theme selection persists across browser sessions ### Adding New Data 1. Update the CSV files with new information 2. Run appropriate scripts from the `scripts/` folder -3. Regenerate cover art if needed using download scripts +3. Regenerate cover art if needed using `download_all_covers.py` +4. Update Wikipedia mappings with `extract_wikipedia_urls.py` ### Contributing - All album descriptions marked "(by Claude)" were AI-generated based on historical research - Original Rolling Stone descriptions preserved where available - Cover art sourced from iTunes API with proper attribution +- Wikipedia links extracted via automated scraping for accuracy ## 📜 License @@ -122,11 +156,11 @@ This project is for educational and research purposes. Album artwork and descrip - **Rolling Stone Magazine** for the original rankings and descriptions - **iTunes API** for high-quality album artwork -- **Wikipedia contributors** for additional album research +- **Wikipedia contributors** for additional album research and accurate page URLs - **Claude AI** for data processing assistance and missing descriptions --- -*Explore the greatest albums of all time and discover how musical tastes and recognition have evolved between 2020 and 2023.* +*Explore the greatest albums of all time with beautiful themes, comprehensive data, and seamless Wikipedia integration. Discover how musical tastes and recognition have evolved between 2020 and 2023.* -🎧 **[Start Exploring →](index.html)** \ No newline at end of file +🎧 **[Start Exploring →](index.html)** | 🎨 **Try Different Themes** | 🔗 **Share Your Favorites** \ No newline at end of file diff --git a/extract_spotify_urls.py b/extract_spotify_urls.py new file mode 100644 index 0000000..e8972e8 --- /dev/null +++ b/extract_spotify_urls.py @@ -0,0 +1,193 @@ +#!/usr/bin/env python3 +""" +Extract Spotify URLs for Top 500 Albums + +This script searches for each album on Spotify using the Spotify Web API +and creates a mapping file similar to the Wikipedia URLs. + +Note: You'll need to set up a Spotify app at https://developer.spotify.com/ +and get your client credentials. +""" + +import csv +import json +import time +import urllib.parse +import urllib.request +import base64 +from typing import Dict, Optional + +# Spotify API credentials (you'll need to get these from Spotify Developer Dashboard) +CLIENT_ID = "your_client_id_here" +CLIENT_SECRET = "your_client_secret_here" + +def get_spotify_access_token() -> Optional[str]: + """Get access token from Spotify API""" + auth_url = "https://accounts.spotify.com/api/token" + + # Encode credentials + credentials = f"{CLIENT_ID}:{CLIENT_SECRET}" + encoded_credentials = base64.b64encode(credentials.encode()).decode() + + headers = { + 'Authorization': f'Basic {encoded_credentials}', + 'Content-Type': 'application/x-www-form-urlencoded' + } + + data = 'grant_type=client_credentials' + + try: + request = urllib.request.Request(auth_url, data=data.encode(), headers=headers) + response = urllib.request.urlopen(request) + result = json.loads(response.read().decode()) + return result.get('access_token') + except Exception as e: + print(f"Error getting access token: {e}") + return None + +def search_spotify_album(artist: str, album: str, access_token: str) -> Optional[str]: + """Search for album on Spotify and return the album URL""" + + # Clean up artist and album names for search + search_artist = artist.replace("&", "and").strip() + search_album = album.replace("&", "and").strip() + + # Remove common prefixes that might confuse search + if search_artist.startswith("The "): + search_artist_alt = search_artist[4:] + else: + search_artist_alt = f"The {search_artist}" + + # Try different search strategies + search_queries = [ + f'album:"{search_album}" artist:"{search_artist}"', + f'album:"{search_album}" artist:"{search_artist_alt}"', + f'"{search_album}" "{search_artist}"', + f'"{search_album}" "{search_artist_alt}"', + f'{search_album} {search_artist}', + ] + + for query in search_queries: + try: + encoded_query = urllib.parse.quote(query) + search_url = f"https://api.spotify.com/v1/search?q={encoded_query}&type=album&limit=10" + + headers = { + 'Authorization': f'Bearer {access_token}' + } + + request = urllib.request.Request(search_url, headers=headers) + response = urllib.request.urlopen(request) + data = json.loads(response.read().decode()) + + albums = data.get('albums', {}).get('items', []) + + if albums: + # Look for the best match + for spotify_album in albums: + spotify_name = spotify_album['name'].lower() + spotify_artist = spotify_album['artists'][0]['name'].lower() + + # Check for exact or close matches + if (album.lower() in spotify_name or spotify_name in album.lower()) and \ + (artist.lower() in spotify_artist or spotify_artist in artist.lower()): + return spotify_album['external_urls']['spotify'] + + # If no perfect match, return the first result + return albums[0]['external_urls']['spotify'] + + # Rate limiting + time.sleep(0.1) + + except Exception as e: + print(f"Error searching for {artist} - {album}: {e}") + time.sleep(1) + continue + + return None + +def main(): + """Main function to extract Spotify URLs""" + + print("Spotify URL Extractor for Top 500 Albums") + print("=" * 50) + + # Check if credentials are set + if CLIENT_ID == "your_client_id_here" or CLIENT_SECRET == "your_client_secret_here": + print("ERROR: Please set your Spotify API credentials in the script!") + print("1. Go to https://developer.spotify.com/") + print("2. Create an app and get your Client ID and Client Secret") + print("3. Replace CLIENT_ID and CLIENT_SECRET in this script") + return + + # Get access token + print("Getting Spotify access token...") + access_token = get_spotify_access_token() + if not access_token: + print("Failed to get access token. Check your credentials.") + return + + print("Access token obtained successfully!") + + # Read the albums data + albums = [] + try: + with open('top_500_albums_2023.csv', 'r', encoding='utf-8') as f: + reader = csv.DictReader(f) + albums = list(reader) + except FileNotFoundError: + print("Error: top_500_albums_2023.csv not found!") + return + + print(f"Found {len(albums)} albums to process") + + spotify_mappings = {} + failed_albums = [] + + for i, album in enumerate(albums, 1): + artist = album['Artist'] + album_name = album['Album'] + + print(f"[{i}/{len(albums)}] Searching: {artist} - {album_name}") + + spotify_url = search_spotify_album(artist, album_name, access_token) + + if spotify_url: + spotify_mappings[album_name] = spotify_url + print(f" ✓ Found: {spotify_url}") + else: + failed_albums.append((artist, album_name)) + print(f" ✗ Not found") + + # Rate limiting to be respectful to Spotify API + time.sleep(0.2) + + # Save progress every 50 albums + if i % 50 == 0: + with open('spotify_urls_mapping_progress.json', 'w', encoding='utf-8') as f: + json.dump(spotify_mappings, f, indent=2, ensure_ascii=False) + print(f"Progress saved. Found {len(spotify_mappings)} URLs so far.") + + # Save the final mappings + with open('spotify_urls_mapping.json', 'w', encoding='utf-8') as f: + json.dump(spotify_mappings, f, indent=2, ensure_ascii=False) + + # Save failed albums for manual review + if failed_albums: + with open('failed_spotify_searches.txt', 'w', encoding='utf-8') as f: + f.write("Albums not found on Spotify:\n") + f.write("=" * 40 + "\n") + for artist, album in failed_albums: + f.write(f"{artist} - {album}\n") + + print(f"\nCompleted!") + print(f"Successfully found Spotify URLs for {len(spotify_mappings)} albums") + print(f"Failed to find {len(failed_albums)} albums") + print(f"Success rate: {len(spotify_mappings)/len(albums)*100:.1f}%") + print(f"\nResults saved to:") + print(f" - spotify_urls_mapping.json") + if failed_albums: + print(f" - failed_spotify_searches.txt") + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/script.js b/script.js index 5062bcf..3d31e31 100644 --- a/script.js +++ b/script.js @@ -5,6 +5,7 @@ let currentPage = 1; const itemsPerPage = 50; let isReversed = false; let wikipediaUrlMappings = {}; +let spotifyUrlMappings = {}; // DOM elements const albumsGrid = document.getElementById('albumsGrid'); @@ -22,6 +23,7 @@ const stats = document.getElementById('stats'); // Initialize the application document.addEventListener('DOMContentLoaded', function() { loadWikipediaMapping(); + loadSpotifyMapping(); loadAlbumsData(); setupEventListeners(); loadTheme(); @@ -40,6 +42,18 @@ async function loadWikipediaMapping() { } } +// Load Spotify URL mappings +async function loadSpotifyMapping() { + try { + const response = await fetch('spotify_urls_mapping.json'); + if (response.ok) { + spotifyUrlMappings = await response.json(); + } + } catch (err) { + console.warn('Could not load Spotify URL mappings:', err); + } +} + // Setup event listeners function setupEventListeners() { searchInput.addEventListener('input', debounce(handleSearch, 300)); @@ -275,6 +289,12 @@ function createAlbumCard(album) { View on Wikipedia → + + + + + Listen on Spotify +