I maintain a Python scraper that collects daily front-page headlines and metadata from Argentine news outlets, supporting sociological research at OSPO (Observatorio de Medios, Facultad de Humanidades, UNMDP). That data feeds monthly analysis of the city’s headlines, plus occasional deep dives into specific topics, and we’re now enriching it with AI, tagging each headline with the actors, topics, and locations involved. It didn’t start on a server I own, it started on AWS Lambda + S3. A few months ago I moved it to a self-managed VPS, and the reasons behind that move ended up more interesting than the move itself.
Why AWS Lambda First
Lambda made sense at the time. No server to provision, no OS to patch, just a function that wakes up on a schedule, scrapes a handful of sites, and writes the results to an S3 bucket. For a research project with an uncertain future, that low commitment was the right call.
Why I Actually Moved
I didn’t move because Lambda was technically wrong for the job, and I didn’t do it to save money either. A VPS isn’t necessarily cheaper than what I was paying on AWS, sometimes it’s about the same, sometimes it’s more. What forced the decision was the free trial period running out. Once I had to choose, I chose to pay for a VPS instead of continuing with AWS. What I was really buying wasn’t a lower bill, it was a flat, predictable price instead of one that changes depending on what I decide to run next. Not having to think twice before trying something new mattered more to me than saving money.
I Already Had Somewhere to Put It
That decision was easy because I wasn’t starting from zero. Picking the VPS provider itself took me a while, comparing options before settling on Netcup, a German hosting provider. I’d already been building my own environment there, documented in a separate server-docs repo, for personal projects, dotfiles, and experiments. Moving the scraper there felt less like a new infrastructure project and more like bringing one more tenant into a house I was already maintaining.
The migration itself wasn’t hard at all. The scraping code moved over, the cron schedule moved with it, and instead of writing to S3, results now sync out via Google Drive (rsync) as a lightweight, good enough backup and access layer.
That last part mattered more than I expected. The researchers using this data aren’t developers. They don’t want an AWS console or S3 credentials, they want a folder of CSVs they can open. Syncing to Google Drive turned “ask me to pull the files” into “they’re already in your Drive,” a small technical change with a bigger effect on how usable the data actually is for the people it’s for. AWS probably has a smoother answer for non-technical access than handing someone raw S3 bucket permissions. I just never needed to find it, because Drive was already the tool my collaborators knew.
Infraestructure Started to Feel Like Mine
It became the home for more than one project
Once the VPS was running my scraper reliably, it stopped being “the machine for one project” and became the place I run things in general. A Telegram bot demo and a handful of other scripts live there now too. Owning the box lowered the barrier for the next automation idea. There was nowhere else that idea would have gone.
The terminal got a lot more comfortable
I open the VPS often enough that I gave it its own tmux shortcut, using mosh for a connection stable enough that flaky wifi (looking at you, Argentina’s Claro) doesn’t kill the session:
bind S if-shell 'tmux has-session -t vps01 2>/dev/null' { switch-client -t vps01} { new-session -d -s vps01 -n mosh "mosh vps01 -- sh -c 'tmux a || tmux new'" new-window -d -t vps01: -c "$HOME/repos/server-docs" select-window -t vps01:1 switch-client -t vps01}One keybinding and I’m inside a live mosh session on the server with server-docs open in a second window. It’s a small thing, but it’s the kind of small thing that decides whether I actually check on a system or just avoid it.
AWS’s real convenience
I want to be fair to Lambda here. Not having to think about the environment at all is a real advantage, and for teams shipping fast or without ops capacity, it’s often the right trade-off. It just wasn’t what I was optimizing for. I was already comfortable owning a Linux box, and honestly, part of the migration turned into pure procrastination getting Neovim properly configured against Debian’s packaged versions of everything. Time that had nothing to do with the scraper and everything to do with enjoying the environment I’d just taken over.
That comfort wasn’t a coincidence. I’d used Debian as my daily desktop OS for about a year before this, so none of it was new. I already knew I’d need to compile a couple of tools from source instead of grabbing them from a repo, and I already had the habit, learned via PEP 668, of not touching global pip install. Putting Debian on the VPS wasn’t a new experiment, just familiar ground.
Watching it run without opening the AWS console
I run Grafana on the VPS, and it gives me a dashboard I actually open, a quick view of whether things are running and healthy. The AWS console never earned that habit from me. It wasn’t uncomfortable in any dramatic way, just enough friction that for something I want to check often, it lost out to a dashboard that’s one bookmark away.
Working on the server from local AI tools
Being able to open the VPS directly from Claude Code or OpenCode in my local terminal has been useful for configuration and troubleshooting, with the obvious caveat that I read every command before it runs against a production box. That habit has saved me from at least one bad idea, and helped me fix a real issue faster than digging through docs alone would have.