323 lines
9.9 KiB
Text
323 lines
9.9 KiB
Text
Loaded 5 API key(s): ['GOOGLE_YOUTUBE_TOKEN', 'GOOGLE_YOUTUBE_TOKEN_v2', 'GOOGLE_YOUTUBE_TOKEN_v3', 'GOOGLE_YOUTUBE_TOKEN_v4', 'GOOGLE_YOUTUBE_TOKEN_v5']
|
|
============================================================
|
|
YouTube Enrichment Script for Heritage Custodians
|
|
============================================================
|
|
API Keys: 5/5 keys available
|
|
Current key: GOOGLE_YOUTUBE_TOKEN (AIzaSyBH...r824)
|
|
Entries directory: data/nde/enriched/entries
|
|
Dry run: False
|
|
Entries to process: 1674
|
|
============================================================
|
|
|
|
Processing: 0000_Q22246632.yaml
|
|
Already fully enriched (113/112 videos, 5 with comments), skipping...
|
|
|
|
Processing: 0001_Q2679819.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0002_Q1978308.yaml
|
|
Already fully enriched (835/835 videos, 203 with comments), skipping...
|
|
|
|
Processing: 0003_Q1258370.yaml
|
|
Already fully enriched (231/230 videos, 40 with comments), skipping...
|
|
|
|
Processing: 0004_Q1888033.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0005_Q81181263.yaml
|
|
Already fully enriched (79/79 videos, 3 with comments), skipping...
|
|
|
|
Processing: 0006_Q81181245.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0007_Q81181227.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0008_Q81181409.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0009_unknown.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0010_Q81181270.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0011_Q81181243.yaml
|
|
Already fully enriched (49/49 videos, 4 with comments), skipping...
|
|
|
|
Processing: 0012_Q81181321.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0013_Q81181217.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0014_Q81181341.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0015_Q81181229.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0016_Q81181377.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0017_harmonium_museum_nederland.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0018_historische_vereniging_carspel_oderen.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0019_de_wieker_meule.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0020_roel_van_der_sleen.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0021_historische_vereniging_nijeveen.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0022_Q1911968.yaml
|
|
Already fully enriched (469/469 videos, 24 with comments), skipping...
|
|
|
|
Processing: 0023_Q22006174.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0024_Q110995931.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0025_Q56461228.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0026_Q2363643.yaml
|
|
Partial enrichment detected (missing comments), re-processing...
|
|
Using existing channel ID: UCHXBVgzI2Cpz-An370a-a3w
|
|
Fetching channel info for UCHXBVgzI2Cpz-An370a-a3w...
|
|
Fetching ALL videos from channel...
|
|
Using uploads playlist: UUHXBVgzI2Cpz-An370a-a3w
|
|
Found 3 videos in uploads playlist
|
|
Fetching comments for all 3 videos...
|
|
Status: SUCCESS
|
|
Channel: Museum Valse Kunst Vledder
|
|
Subscribers: 4
|
|
Videos fetched: 3
|
|
|
|
Processing: 0027_Q19832258.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0028_aold_daoln.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0029_Q110995917.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0030_Q14629603.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0031_onderduikersmuseum_de_duikelaar.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0032_oudheidkamer_zuidwolde.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0033_Q81181201.yaml
|
|
Already fully enriched (835/835 videos, 203 with comments), skipping...
|
|
|
|
Processing: 0034_rolder_historisch_gezelschap.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0035_ruinen_1865.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0036_historie_van_ruinerwold.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0037_Q1345886.yaml
|
|
Already fully enriched (9/9 videos, 1 with comments), skipping...
|
|
|
|
Processing: 0038_Q2429034.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0039_Q108369683.yaml
|
|
Already fully enriched (59/59 videos, 4 with comments), skipping...
|
|
|
|
Processing: 0040_museum_contemporary_tibetan_art.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0041_asser_historische_vereniging.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0042_asser_historische_vereniging_2.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0043_historische_vereniging_gasselte.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0044_Q124789020.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0045_Q59961895.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0046_cultuur_eus_tromp_de_vries.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0047_Q30277559.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0048_Q791082.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0049_Q110907349.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0050_Q20962412.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0051_Q2739162.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0052_Q2155567.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0053_huis_van_hilde.yaml
|
|
ERROR: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/new:lxml.etree._ElementUnicodeResult'
|
|
in "data/nde/enriched/entries/0053_huis_van_hilde.yaml", line 517, column 16
|
|
|
|
Processing: 0054_Q2777409.yaml
|
|
Already fully enriched (518/518 videos, 170 with comments), skipping...
|
|
|
|
Processing: 0055_Q121221960.yaml
|
|
Already fully enriched (86/86 videos, 14 with comments), skipping...
|
|
|
|
Processing: 0056_Q110985372.yaml
|
|
Already fully enriched (52/52 videos, 12 with comments), skipping...
|
|
|
|
Processing: 0057_geschiedschrijving_dronten.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0058_vrienden_van_urk.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0059_Q123380579.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0060_Q18286705.yaml
|
|
Partial enrichment detected (missing comments), re-processing...
|
|
Using existing channel ID: UCRK54Xrx4tk5KOtZRK7yXwA
|
|
Fetching channel info for UCRK54Xrx4tk5KOtZRK7yXwA...
|
|
Fetching ALL videos from channel...
|
|
Using uploads playlist: UURK54Xrx4tk5KOtZRK7yXwA
|
|
|
|
⚠️ Quota exceeded detected, rotating key...
|
|
|
|
⚠️ Quota exceeded for GOOGLE_YOUTUBE_TOKEN_v2, rotating to next key...
|
|
Retrying with GOOGLE_YOUTUBE_TOKEN...
|
|
|
|
Processing: 0060_Q18286705.yaml
|
|
Partial enrichment detected (missing comments), re-processing...
|
|
Using existing channel ID: UCRK54Xrx4tk5KOtZRK7yXwA
|
|
Fetching channel info for UCRK54Xrx4tk5KOtZRK7yXwA...
|
|
Fetching ALL videos from channel...
|
|
Using uploads playlist: UURK54Xrx4tk5KOtZRK7yXwA
|
|
Found 7 videos in uploads playlist
|
|
⚠️ Key also exhausted, rotating...
|
|
|
|
⚠️ Quota exceeded for GOOGLE_YOUTUBE_TOKEN, rotating to next key...
|
|
Retrying with GOOGLE_YOUTUBE_TOKEN_v3...
|
|
|
|
Processing: 0060_Q18286705.yaml
|
|
Partial enrichment detected (missing comments), re-processing...
|
|
Using existing channel ID: UCRK54Xrx4tk5KOtZRK7yXwA
|
|
Fetching channel info for UCRK54Xrx4tk5KOtZRK7yXwA...
|
|
Fetching ALL videos from channel...
|
|
Using uploads playlist: UURK54Xrx4tk5KOtZRK7yXwA
|
|
Found 7 videos in uploads playlist
|
|
Fetching comments for all 7 videos...
|
|
Status: SUCCESS
|
|
Channel: BunkermuseumVlieland
|
|
Subscribers: 40
|
|
Videos fetched: 7
|
|
|
|
Processing: 0061_Q13137171.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0062_Q2345721.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0063_Q2530771.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0064_Q14943525.yaml
|
|
Partial enrichment detected (missing comments), re-processing...
|
|
Using existing channel ID: UCvjh45L4T4-wqWykQg76Afg
|
|
Fetching channel info for UCvjh45L4T4-wqWykQg76Afg...
|
|
|
|
⚠️ Quota exceeded detected, rotating key...
|
|
|
|
⚠️ Quota exceeded for GOOGLE_YOUTUBE_TOKEN_v3, rotating to next key...
|
|
Retrying with GOOGLE_YOUTUBE_TOKEN_v4...
|
|
|
|
Processing: 0064_Q14943525.yaml
|
|
Partial enrichment detected (missing comments), re-processing...
|
|
Using existing channel ID: UCvjh45L4T4-wqWykQg76Afg
|
|
Fetching channel info for UCvjh45L4T4-wqWykQg76Afg...
|
|
Fetching ALL videos from channel...
|
|
Using uploads playlist: UUvjh45L4T4-wqWykQg76Afg
|
|
Found 5 videos in uploads playlist
|
|
Fetching comments for all 5 videos...
|
|
Status: SUCCESS
|
|
Channel: Keatsmuseum Franeker
|
|
Subscribers: Hidden
|
|
Videos fetched: 5
|
|
|
|
Processing: 0065_Q13138176.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0066_Q56459741.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0067_museum_t_fiskershuske.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0068_museum_t_fiskershuske_2.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0069_Q13137174.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0070_Q17517652.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0071_Q124652958.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0072_Q3457274.yaml
|
|
No YouTube URL found, skipping...
|
|
|
|
Processing: 0073_Q2011870.yaml
|
|
Partial enrichment detected (missing comments), re-processing...
|
|
Using existing channel ID: UCSi3ozc4oHkCwjxQ-8x23uQ
|
|
Fetching channel info for UCSi3ozc4oHkCwjxQ-8x23uQ...
|
|
Fetching ALL videos from channel...
|
|
Using uploads playlist: UUSi3ozc4oHkCwjxQ-8x23uQ
|
|
|
|
⚠️ Quota exceeded detected, rotating key...
|
|
|
|
⚠️ Quota exceeded for GOOGLE_YOUTUBE_TOKEN_v4, rotating to next key...
|
|
Retrying with GOOGLE_YOUTUBE_TOKEN_v5...
|
|
|
|
Processing: 0073_Q2011870.yaml
|
|
Partial enrichment detected (missing comments), re-processing...
|
|
Using existing channel ID: UCSi3ozc4oHkCwjxQ-8x23uQ
|
|
Fetching channel info for UCSi3ozc4oHkCwjxQ-8x23uQ...
|
|
⚠️ Key also exhausted, rotating...
|
|
|
|
⚠️ Quota exceeded for GOOGLE_YOUTUBE_TOKEN_v5, rotating to next key...
|
|
No more keys available, stopping.
|
|
|
|
============================================================
|
|
Summary
|
|
============================================================
|
|
Entries processed: 1674
|
|
Successfully enriched: 3
|
|
Skipped (no YouTube / already done): 69
|
|
Errors: 1
|