Category: Automation / Data Collection Difficulty: Medium Setup time: 20-40 minutes Last tested: January 2025

THE PROBLEM

I wanted to find data quickly from websites. Automatically. Without doing things manually.

But I wanted to do it the right way - without crossing legal lines, without breaking terms of service, staying within what's acceptable.

I needed a way to:

• Automate browsing
• Extract information from websites
• Get real-time notifications about listings
• Do it at scale, not one page at a time

THE SOLUTIONS I TRIED

I explored two main approaches:

1. Playwright with Headless/Headful browsers - Code-based automation
2. UIVision - Record and replay automation

Both have their place. Both have limitations.

METHOD 1: PLAYWRIGHT + HEADLESS/HEADFUL

What is Playwright?

Playwright is a tool that lets you control a web browser with code. You write scripts, and the browser does exactly what you tell it - navigate, click, type, extract data.