Data engineering2025
Estimating asking prices from 500+ listings collected with n8n
An n8n workflow collects new Idealista listings from my area of Spain, and a linear baseline estimates their asking prices. The hard part is the target: an asking price is a seller's decision, not a valuation.
- Role
- Personal project, end to end
- Timeline
- October 2025
- Stack
- n8n, Python, BeautifulSoup, Pandas, scikit-learn
- Scope
- 500+ listings, one area of Spain
- Model
- Linear regression baseline
- n8n
- Web scraping
- Data pipelines
- Python
- BeautifulSoup
- Pandas
- scikit-learn
- Feature engineering
- +1
- Problem
- I follow house prices in my area of Spain. Idealista, a widely used property portal, has the listings, but there is no clean dataset for a local market like this one, and browsing by hand does not build one.
- Approach
- An n8n workflow collects only new listings, one page at a time, with a random pause after each. Python (BeautifulSoup, Pandas) turns each listing into features, including keyword rules on the description, and fits a linear regression.
- Outcome
- 500+ listings and a baseline with an observed error of roughly ±25k. The error traces back to the target itself and to what listings leave out.
Collection
Each run fetches only what is new
There was no dataset to download, so the first job was a collector, built in n8n as a personal research project. Each run reads the search results for listing IDs, drops every ID already stored, and fetches the rest one at a time, with a random pause after each listing.
The design keeps the load small: a listing already in the sheet is never fetched again, and a failed fetch ends the run instead of writing a broken row.
01
Search
Cookies · HTTP Request · HTML
- Cookies set once per run
- Listing IDs read from search results
02
Deduplicate
Get rows · Code
- IDs already in the sheet dropped
- Only new listings go on
03
Fetch
Loop Over Items · HTTP Request · HTML
- One listing page per iteration
- Page content extracted
04
Check and store
Switch · Code · Append row
- Errors stop the run
- Features appended as one row
05
Pause
Rand Delay · Wait
- Random delay
- Then the next listing
The sheet does two jobs: it holds the dataset, and it is the list of IDs to skip on the next run.
Features
The structured fields are the easy part
Every listing carries a few structured fields: floor area, bedrooms, bathrooms, floor and an approximate location. BeautifulSoup pulls them out of the listing's HTML, and that part is routine.
Crucial information often sits somewhere else. Whether a home needs renovation or is newly built, whether it is occupied or missing a permit: that lives in the free-text description, when it is mentioned at all. Many listings also leave fields empty.
From listing to feature row
| Signal | Where it appears | What I did |
|---|---|---|
| Floor area, bedrooms, bathrooms, floor, approximate location | Structured fields | Parsed from the HTML |
| Stated condition: “to renovate”, “renovated”, “new” | Description | Keyword rules into coarse categories |
| Distance to the coast | Neighbourhood name, mentions of “sea views” | Approximate proxy feature |
| Missing values | Many listings | Standardised columns, median/mode imputation |
| Occupied, missing permits | Description, if mentioned | Not captured |
| Actual interior condition | Photos | Not captured |
| Real sea view, orientation, noise | Hard to label automatically | Not captured |
Rows marked “not captured” are information the model never sees.
Every label in this dataset is a price someone chose to ask. The model learns what sellers ask, not what homes are worth.
Model
A linear baseline, on purpose
I fitted a linear regression with scikit-learn. The features are limited, and the target is soft: owners and agencies set asking prices, so there is no single ground-truth valuation to label each home with.
In that setting, an interpretable baseline is the honest first step. It shows how much the structured data explains on its own, without a flexible model squeezing patterns out of noisy labels.
Error
Where the ±25k comes from
- listings collected
- 500+
- One area of Spain, via the n8n workflow
- observed error
- ±25k
- Linear baseline on asking price, in euros
Dataset and baseline as of October 2025.
That error is expected for this target. Sellers value similar homes differently, and the model only ever sees what they ask.
The rest is information a listing does not carry in a usable form. This is a beach area, so closeness to the sea and a real sea view have a big impact on price, and my distance-to-coast proxy is only approximate. Orientation, noise and the actual state of the interior matter too, and all of them are hard to label automatically.
So this is a baseline, not a valuation tool. What lasts is the pipeline: a repeatable, local view of the market with a transparent model on top.
Next
What would reduce the error
- Read the photos. Use computer vision on listing images to infer interior condition and renovation needs, which the text only hints at.
- Keep collecting. Extend coverage across time and across the area.
- Add complexity when it pays. Try regularised or non-linear models once new features justify them.
Next case study
Building Polymarket trading systems until the evidence said stop
Related
Web product2026Live
Moduly Homes
A live comparator of 1,137 prefab home models, with a source for every figure.
modulyhomes.comData engineering
To predict Polymarket outcomes, I first had to build the dataset
I set out to predict long-horizon Polymarket markets and ended up building the dataset: a resumable collector over three APIs, with holder-level features.
Machine learning2025Published
Gym exercise recognition from IMU data
A CNN-LSTM that recognises gym exercises from wrist-worn motion sensors.
Get in touch
Working on something similar? I am happy to compare notes.