Mesmer Lord (@mesmerlord)

Is web scraping legal? The social media cases so far

Originally published at https://lurkapi.com/blog/is-web-scraping-legal In the US, mostly yes, if the pages are public and you aren’t logged in. Courts have ruled that reading a public page isn’t hack

3h ago · 8 min readlurkapi.hashnode.dev

Why youtube-transcript-api gets blocked on servers, and what works

Originally published at https://lurkapi.com/blog/youtube-transcript-api-blocked When youtube-transcript-api works on your laptop and raises RequestBlocked on AWS, a VPS, Colab or a CI runner, your co

3h ago · 7 min readlurkapi.hashnode.dev

403 Forbidden in Python requests: which bot wall sent it

Originally published at https://lurkapi.com/blog/scraper-403-forbidden If Chrome opens a page and your Python script gets 403 Forbidden, it’s usually a bot wall in front of the site that refused you:

3h ago · 10 min readlurkapi.hashnode.dev

Mesmer Lord

About

Available for

Mesmer Lord's blogs

Is web scraping legal? The social media cases so far

Why youtube-transcript-api gets blocked on servers, and what works

403 Forbidden in Python requests: which bot wall sent it

Mesmer Lord

About

Available for

Mesmer Lord's blogs

Is web scraping legal? The social media cases so far

Why youtube-transcript-api gets blocked on servers, and what works

403 Forbidden in Python requests: which bot wall sent it

Mesmer Lord

About

Available for

Mesmer Lord's blogs

Is web scraping legal? The social media cases so far

Why youtube-transcript-api gets blocked on servers, and what works

403 Forbidden in Python requests: which bot wall sent it

Search Hashnode

Mesmer Lord

About

Available for

Mesmer Lord's blogs

Is web scraping legal? The social media cases so far

Why youtube-transcript-api gets blocked on servers, and what works

403 Forbidden in Python requests: which bot wall sent it