Here's a quick video to show you how it works: https://www.loom.com/share/8d27d0f9e0f34c77a81b0ac770151c12

A couple weeks ago, we needed a way to automatically find 3D assets from the internet for one of our work contracts. We didn’t need an AI to choose all the steps for us autonomously — we knew generally what the flow to find items should be and just needed some intelligence to handle different webpage formats.

Playwright couldn’t easily generalize to the different sites we wanted to search over and Claude computer use was tough to use — it was slow, expensive, struggled to click on the correct web elements, and there was no way to correct its actions if it failed at step 20 of our workflow.

So we built a vision-based solution using vision language models that sits on top of Playwright. In our SDK, building a function that can universally search websites is as easy as this:

    from simplex import Simplex
    simplex = Simplex(api_key="api_key")
    
    def search(website, search_query):
        simplex.goto(website)
        simplex.click("the search bar")
        simplex.type(search_query)
        simplex.press_enter()
        
    search("mit.edu", "alyssa p hacker")

We'd love feedback!