screenpipe #006

⬇️ download mac m app
⬇️ download windows app
give us a ⭐️(11.5k) to help the algorithm

hey louis & matt here

btw, in this newsletter, we share new features, bug fixes, work in progress, and what's next.

as a reminder, screenpipe is an open source library and an app to record data from your screens & mics and pass it into a llm. an ai powered by what you've seen, said, or heard:

✨ what's new

🔍 search

you can now search your desktop history the old-school way—by keywords, dates, apps, or even that random window you accidentally closed

it's basic for now and still a work in progress. at the same time, we're going to make the chat more intuitive and require less prompt engineering in the future

🪟 window, app, tab filtering

record specific windows, apps, tabs, or filter them out

for example:

• filter out all windows that contain ".env" in the title, good for devs
• ignore all windows that contain "bit". it will ignore all windows and tabs containing "bit" like "bitwarden", "bittorent", etc.
• or select "chrome" if you want to record only google chrome

🔌 new pipes (plugins)

• security pipe that warns user about potentially dangerous actions, e.g. phishing e-mails (demo)
• put ai guardrails around humans: llama 3.1 blocking user keyboard and mouse when it identifies unwanted behavior, proof-of-concept (demo)

⭐️ other new features

• send desktop notifications from screenpipe at your choice
• macos: no permission issues anymore, smooth installation experience
• screenpipe plugins: pipes, pass configs, write typescript
• multi-monitor recording & ocr now in settings in ui and cli
• new voice detection implementation with silero vad
• no windows defender alerts
• linux wayland fix (wip)
• known issues: sometimes macos audio output crashes, work around is to create a virtual audio device that contains the system audio or uses restart interval settings which forces a restart of the process

🔜 next

• use case: automatic summaries for audio/text conversation by speaker
• use case: highlight grammar suggestions on the screen in real-time
• more examples of preventive actions by llm, notifications + taking over control
• more advanced desktop control in pipes, more examples
• installation, stability & performance cross-platform, fix known issues
• search: more relevancy, less redundant information
• new models: brand new local speech to text model - silero
• new models: add new ocr engine (esp. for linux)
• security: mp4 encryption at rest
• storage: reduce storage required by half using h265 encoding
• ux: automatic audio device switch
• ux: more reliable interface, specific interface for audio summaries, global shortcut, cursor-alike chat, shortcuts, etc.
• ux: more customizable app ai settings (use your own openai api compatible URL)
• extensions: plenty of plugins you can install in a click (or build yourself) to get the most out of your data
• extensions: use native api (control your computer, display on the screen, etc.), real-time data streaming, high level abstraction in pipes

btw we're running a bunch of paid bounties (up to $200 atm) to make screenpipe even better, just go to github issues and check issues with label "bounty"

• the app is still in alpha and we've fixed tons of bugs, however, we're releasing daily updates to fix them, along with new features. we're a two-person team, but we have open source contributors joining, and we would be happy to welcome more! ☺️🙏

links

• download mac m app
• download windows app

take care,
screenpipe

follow us:
x
youtube
discord

wanna chat?

You are receiving this email because you opted-in to receive updates from Mediar, Inc
Mediar, Inc, 2 Marina Blvd B300, San Francisco, CA 94123