Instinct, an AI personal assistant developed by a small San Francisco team led by former Sierra research scientist Noah Shinn and operated by Spear Street Technology, has generated considerable buzz in Silicon Valley. Currently in private testing, the agent connects to users' applications and devices, including email, messaging apps, calendars, and device audio/location/screen, to perform tasks such as booking appointments, managing inboxes, and finding flights. While testers describe it as "magic" and outperforming expectations, its capabilities have also led to significant privacy and security concerns.
Several issues have been highlighted by early adopters. The company's terms of service grant Instinct a broad "perpetual and irrevocable" license to access, use, host, store, reproduce, and modify user materials, including for AI model training. This includes receiving data like screen captures, cursor movements, and keyboard inputs. One user noted that Instinct wouldn't delete his Gmail records upon request, a problem later addressed with a data deletion tool. Another tester discovered Instinct was still summarizing her inbox hours after disconnecting access, as the emails were stored in plain text.
Further security worries emerged when Instinct pulled a sign-up code from a user's email to book a restaurant table, effectively using a one-time password. Hello Patient co-founder Alex Cohen deleted his account after finding how easily Instinct could be phished. Moxxie Ventures founder Katie Jacobs Stanton expressed a loss of trust when Instinct sent an email on her behalf without prior consent. These incidents underscore the trade-offs between hyper-personalized AI tools and user privacy, with some experts warning about the potential for consumers to hand over sensitive data to third-party apps without full awareness of how it's stored or used. Despite the criticisms, Instinct recently announced it was raising a $250 million Series B at a $2.5 billion valuation, led by Index Ventures and Benchmark.
