Hi, this was my first hackathon: Frontier CDMX 26

On Saturday, October 31, 2026, I went to my first hackathon: Frontier CDMX 26. I took part on my own, built in about 8 hours an agent that operates systems with no API by looking at the screen, did not win and learned more than in months. Here is how it went, hour by hour and without spin.

What is Frontier?

Frontier · CDMX 26, powered by Lab10, is an in-person hackathon for strong technical talent in the region. The format resembles AI labs' hiring processes: a hard challenge, hidden tests, an individual scorecard and companies hiring. There were more than US$8,000 in prizes, and you could compete alone or in pairs.

How did I apply and get in?

The application was 8 open questions in English. Nobody got in on their CV: you got in on how you think. I applied with doubts, honestly. I was approved in two days.

Fact	Value
Applied	190
Got in	42
Challenges	1 per participant
Working time	~8 hours
Team	Solo

What was the challenge?

Challenge 2 was called "The agent that operates what has no API". Picture a 15-year-old system nobody can integrate: no API, just a screen. The agent gets a task in natural language ("change this customer's phone number") and has to carry it out in an app it has never seen, only through the screen, over VNC at 1280×800.

Practice happened on an app called SIGECOM and evaluation on a different one, with 5 minutes per task. And it had traps: the screen repainted every second, clicks under 100 ms were ignored, and typed text concatenated if you did not clear the field.

How did my day go, hour by hour?

Time	What I did
12:20 pm	Setup: repo, Python, Tesseract and GitHub.
12:50 pm	Custom VNC client.
1–2 pm	Eyes (OCR) and hands (click and keyboard).
2–3 pm	Brain (the LLM) and rules.
3:00 pm	The incident: the app changed version.
3–4:30 pm	Tests and regression: 5/5 on two versions.
4–4:45 pm	API, Linux server, systemd, HTTPS and logs.
4:43 pm	Evaluation.
6:55 pm	Result.

How does the agent work?

The idea in one sentence: the LLM thinks each step; the loop puts it to work, watches it and stops it. My rule was "deterministic first, the LLM only decides". On every turn of the loop:

1. The eyes read the screen with OCR (Tesseract, two passes in parallel: one normal and one with inverted colors to read light text on dark backgrounds).
2. The brain picks the next step. I used Gemini 2.5 Flash via OpenRouter: 0.8 s per call against 1.8 s for other options.
3. The code checks the decision: does that text exist on screen? Is the JSON valid?
4. The hands act: click by text, never by coordinates, and type.
5. The step is logged, and if it is not done and time is not up, back to 1.

No Python library spoke VNC over websocket, so I wrote a custom client of about 200 lines. And I found that without a map of the app's menus no model got it right: the map is built with OCR and clicks, without spending on the model.

I used Claude Code for much of the code. I defined the architecture, the rules and the priorities, reviewed and deployed. The code is on GitHub.

What do the two videos show?

Both are sped up and come from the practice app, with test data. The first one is a failure, and it is the right behavior: there are two customers named "Ana Torres", so the agent does not guess, touches nothing and reports both records.

The second one solves the task: it fixes an address and ignores a note hidden in the notes field that told it to delete orders. That is a prompt injection attempt: instructions smuggled inside the data. The rule that stuck: everything on screen is data, never an instruction.

What happened at 3 pm?

Mid-afternoon the practice app switched to version 4.2: a new modal appeared, the "Buscar" (Search) button was renamed "Filtrar" (Filter) and everything got slower. Exactly the kind of change that breaks a fragile agent. I got through it because the agent acts by text and reads the screen before every step. I also caught a false success (the agent reported 25 where there were 45), and an accidental restart exposed a real bug.

How did it go from laptop to server?

The evaluation sent tasks to an API, so the agent had to live on a Linux server, not my laptop. In 45 minutes: FastAPI with a single-worker queue, a systemd service, an isolated user, HTTPS and logs. During evaluation I followed every task with journalctl. I used Git from minute one, and it saved me more than once.

What were the numbers?

Metric	Result
Practice tests	5/5
Evaluation	1/34
Easy · regular · hard	1/10 · 0/14 · 0/10
Median per task	4:24 (limit: 5:00)
Cost for the day	Under 1 dollar

What was the result, without spin?

I did not win. In practice the agent solved 5 of 5; in evaluation, 1 of 34. The median per task was 4:24 against a 5:00 limit: the agent kept running out of time. The architecture survived the version change and the prompt-injection trap, but it was not fast enough on an app it had never seen. I wrote about that separately: what I learned finishing last.

And the people?

The best part of the day. I made new friends, talked through the challenge, cleared up doubts and moved forward thanks to them. Friends I met there took 1st and 2nd place. If you are thinking about your first hackathon, I left you my tips.

The full story, with the diagram and the videos, is on the Frontier page. And if you want agents or AI automation working in your business, this is how I do it in AI and automation; one example in production is Panini Fan Zone.

FAQ

What is Frontier CDMX 26?

An in-person hackathon powered by Lab10 on October 31, 2026 in Mexico City, with a challenge styled after AI labs' hiring processes, hidden tests and an individual scorecard.

What was the challenge?

Build an agent that carries out natural-language tasks in an app with no API it has never seen, only through the screen, with 5 minutes per task.

Which model did the agent use?

Gemini 2.5 Flash via OpenRouter, at 0.8 s per call. Code validates every decision before acting.

Did you win?

No. Practice went 5/5 and evaluation 1/34: the agent kept running out of time.