screenpipe #006

⬇️ download mac m app
⬇️ download windows app
give us a ⭐️(11.5k) to help the algorithm

hey louis & matt here

btw, in this newsletter, we share new features, bug fixes, work in progress, and what's next.

as a reminder, screenpipe is an open source library and an app to record data from your screens & mics and pass it into a llm. an ai powered by what you've seen, said, or heard:

✨ what's new

🔍 search

you can now search your desktop history the old-school way—by keywords, dates, apps, or even that random window you accidentally closed

search interface

it's basic for now and still a work in progress. at the same time, we're going to make the chat more intuitive and require less prompt engineering in the future

🪟 window, app, tab filtering

record specific windows, apps, tabs, or filter them out

window filtering

for example:

filter out all windows that contain ".env" in the title, good for devs
ignore all windows that contain "bit". it will ignore all windows and tabs containing "bit" like "bitwarden", "bittorent", etc.
or select "chrome" if you want to record only google chrome

🔌 new pipes (plugins)

security pipe that warns user about potentially dangerous actions, e.g. phishing e-mails (demo)
put ai guardrails around humans: llama 3.1 blocking user keyboard and mouse when it identifies unwanted behavior, proof-of-concept (demo)

⭐️ other new features

send desktop notifications from screenpipe at your choice
macos: no permission issues anymore, smooth installation experience
screenpipe plugins: pipes, pass configs, write typescript
multi-monitor recording & ocr now in settings in ui and cli
new voice detection implementation with silero vad
no windows defender alerts
linux wayland fix (wip)
known issues: sometimes macos audio output crashes, work around is to create a virtual audio device that contains the system audio or uses restart interval settings which forces a restart of the process

🔜 next

use case: automatic summaries for audio/text conversation by speaker
use case: highlight grammar suggestions on the screen in real-time
more examples of preventive actions by llm, notifications + taking over control
more advanced desktop control in pipes, more examples
installation, stability & performance cross-platform, fix known issues
search: more relevancy, less redundant information
new models: brand new local speech to text model - silero
new models: add new ocr engine (esp. for linux)
security: mp4 encryption at rest
storage: reduce storage required by half using h265 encoding
ux: automatic audio device switch
ux: more reliable interface, specific interface for audio summaries, global shortcut, cursor-alike chat, shortcuts, etc.
ux: more customizable app ai settings (use your own openai api compatible URL)
extensions: plenty of plugins you can install in a click (or build yourself) to get the most out of your data
extensions: use native api (control your computer, display on the screen, etc.), real-time data streaming, high level abstraction in pipes

btw we're running a bunch of paid bounties (up to $200 atm) to make screenpipe even better, just go to github issues and check issues with label "bounty"

the app is still in alpha and we've fixed tons of bugs, however, we're releasing daily updates to fix them, along with new features. we're a two-person team, but we have open source contributors joining, and we would be happy to welcome more! ☺️🙏

links

take care,
screenpipe

wanna chat?

You are receiving this email because you opted-in to receive updates from Mediar, Inc
Mediar, Inc, 2 Marina Blvd B300, San Francisco, CA 94123

📺 screenpipe #006 | search interface, 600 mb 10% CPU, window filtering