This sounds an awful lot like Microsoft's Recall, only implemented in the browser.
Granted, there have been a lot of times I have trouble finding a website in my history, open tabs or even bookmarks, so I could potentially see how that might be advantageous as long as I was in a situation where I had a second browser for "non-work" related tasks, or this was strictly prohibited in in-private mode.
>4. Find webpages you previously visited
>For those frustrating instances when you want to jump back into a past project but don’t want to scroll through your history to find an important website you previously visited, soon you’ll be able to use Gemini in Chrome to recall it for you. Once launched, you can try prompts like “what was the website that I saw the walnut desk on last week?” or “what was that blog I read on back to school shopping?”
As for their "agentic browsing assistant", I don't have much trouble adding stuff to my shopping cart or other minor tasks. I'm still waiting on that 'Google Duplex' [1] feature they announced years ago that claimed Google would make phone calls for me to make appointments and etc. Make a doctor's appointment? Dispute a charge on my bill? That's what I want.
> As for their "agentic browsing assistant", I don't have much trouble adding stuff to my shopping cart or other minor tasks.
It looks like it is capable of more complex tasks than that including things like making a comparison table of products based on your criteria.
To extend the grocery example, it would be impressive if it could building a shopping cart and multiple stores so you can chose the one with the best total price/availability.
I find that Chrome has a fairly crippled history by default (worse than any other browser I've ever used). It's so bad that I ended up installing a history extension. Works much better.
I mostly have trouble keeping too many browser tabs open on mobile. Granted, I use Brave & it now organizes closed tabs. On desktop, it has a similar Ai feature for tab management to the one Google described, but it's still not great.
I'd honestly appreciate some kind of AI tab management, history/bookmarks saving, summarizing & organizing that would put my old tabs to some kind of reading list that would remind me what I never closed down the line, archive the links I visited & my bookmarks incase of linkrot they would still be saved. Make sure if I was writing a comment on Reddit or similar site, saved it as a draft, etc, etc. That kind of "Smart" browser management system, that I could preferably run myself or had some privacy guarantees (for whatever they're worth) would definitely something I'd consider paying for.
Part of me is actually excited for the acceleration of the end of the web, I even wrote https://punkx.org/jackdoe/zero.txt few days ago in frustration.
The sooner it ends the better.
I really don't want to see "summarization" tokens of other people's tokens.
It is astonishing that the word “privacy” appears zero times in this announcement. There have been repeated controversies over exactly how Google sees just the URL I visit. Now they want to see the entire contents of multiple browser tabs?
Yikes! Given the inherent threat of prompt injection, using the weakest available version of Gemini seems like a particularly bad idea.
Not that even the strongest models are 100% effective at spotting prompt injection attacks, but they have way more of a fighting chance than Gemini nano does.
You could contort the threat model such that prompt injection is something to worry about with a local model operating on local data and serving local results, sure.
What’s really bugging me is they didn’t think it was interesting to even touch on that point in the big announcement. Contrast Apple making a big deal about private cloud compute before it even really does anything.
Yeah it’s insane they’ve totally ignored the privacy issue. Either they’re doing everything on device, which I doubt, or this is the biggest privacy disaster ever waiting to happen.
Sometimes I wish companies would stop forcing AI features down our throats and putting them just everywhere. At least I hope I can properly disable all of this. I don't need an AI agent scanning everything behind my back.
AI is just the current incarnation of the hype train cycle.
I've never been a big fan of smart phones and I remember in the early 2010s the "mobile revolution" was in full tilt and it even impacted the Linux experience. I ended up switching from Ubuntu to Mint because they went all in on "mobile + touch-screens are the future!" and released this god awful UI update that was reminiscent of Windows 8.
We need business to drive innovation ... but there is bad with the good (and vice versa - we shouldn't forget that either). When something gets "hot" the business world will always go all in on the trend and "force" it down everyone's throats. It's driven partly by fear: "If I don't offer this to my customers, my competitors will and I will fail." The rest is the normal pursuit of profit, which isn't a bad thing IMO but it means there's a lot of: "There's a pie here and if we don't get our slice someone else will."
It looks like [0] access to gemini will only be for subscribers, given that it costs them money. This is, "of course", distinct from ai mode [1] in google search that happens from the address bar in chrome. The first video implies that the difference involves throwing the current web page into the query as context so someone can ask "is this recipe gluten-free?" on a recipe webpage.
Taking a quick test spin. Seems to be enabled (on mac) by the existing "Gemini in Chrome" extension but requires a Chrome update. Which has a global shortcut (across all mac apps) of "Ctrl+g" (this can be changed). The additional AI features seem to come from the tab content's integration into the Gemini console (previously (I believe) the Gemini extension was merely a thin console to the Gemini service)).
The direct tab integration works by first showing the Gemini console (ctrl-g or Gemini icon in the mac system tray) where there is then a 'Share current page' icon below the text view.
This adds a blue border around the chrome window indicating the current page can be shared with chrome. It's not clear, but I believe the page content is only shared once a prompt is made intending to use the page's content. However, the share-enablement remains enabled for all tabs (all tab windows will have the blue border) until turned off in the Gemini consol. Again, it's not clear if just merely browsing these tabs will automatically share that content with Google.
The Gemini integration does not perform actions (can't ask it to do stuff directly with the site's content (navigate, click buttons, etc).
But direct summarization works well (try it on an HN comment page or news article).
Overall, I like this feature as long as I understand what and when things are being shared and ability to turn off easily.
>It's not clear, but I believe the page content is only shared once a prompt is made intending to use the page's content.
If it's only processing page content as the user requests it, then how would feature 4 "Find webpages you previously visited" work? Seems like it would need to process everything in order to enable prompt-based content recall.
Anthropic's "Claude for Chrome" pilot from a few weeks back spent most of the announcement article talking about security and threats from prompt injection: https://www.anthropic.com/news/claude-for-chrome
This announcement from Chrome themselves hardly mentions that at all.
> In the coming months, we’ll be introducing agentic capabilities to Gemini in Chrome. These will let Gemini in Chrome handle those tedious tasks that take up so much of your time, like booking a haircut or ordering your weekly groceries.
Can I block it as a site host? (Please don't respond about how I shouldn't want to and isn't it just like some other usecase that I'd obviously want not to block)
Some of these features would be nice to have, but I'm not sure I even want my browser to have these capabilities unless there is a mechanism to keep it all local. This is a monumental change to the amount of user data Google can collect via Chrome.
> ...unless there is a mechanism to keep it all local
I'm not sure the Google is a trustworthy party [0] to believe when they give you some hard-to-find option to keep data local and not use it for user profiling and ad targeting. Google is essentially a data mining business. Some opportunities are simply hard for them to resist.
It is like the rug is pulled beneath our feet. When I was in college I could get a handle of what stuff is worth sharing and what isn’t. Now all of my data from before has become a liability.
Automating tedious tasks is great, as long as it's reliable. We know how to build reliable integrations and reliable automations. Making chat bots a page and click buttons it thinks will do the right thing is never gonna be reliable.
This seems huge to me. As in: the initial release of Google Chrome huge. I don’t know if they can really pull off things they showed but if they do I’m sure this will be a massive success. Which is pretty sad considering the privacy implications for this. Imagine how much more data they will have on everyone. Scary.
A lot of these new AI features can be helpful as long as the users' control the data. Is there an alternate world where Google might become the good guy here? In this world, as might be expected, the company making the worlds best spyware, wants to expand its spying.
Not a comment re: AI in Chrome. I did click into the video to hopefully get an understanding of the features,. but sadly did not. Sorta got an idea, but the jump cuts, super-quick overviews, trying to identify everyone speaking, etc. E.g. ...something about tabs... ...used to take 20 minutes, now seconds... ...working with OTHER Google products (WTH)... Hey Google, make a simpler, more accessible video that is "just the facts mam". Dozens of jump cuts in a few minutes is disconcerting, and may prove dangerous to some.
The interesting part about this variant is, that it's actually happening in my browser. I can't see how else this is going to happen, for various real world reasons. This feels actually tangible and potentially useful. With ChatGPT I am just confused about when/why I would press "Agent".
So they integrate Gemini to summarize open web pages and consolidate all your open tabs into summaries. (Open lot's of pages, then summarize them all.) You can search your history with natural language and type Gemini queries directly into the address bar.
This will give them a cognitive profile of you: reading comprehension, decision-making patterns, knowledge gaps, etc.
When the webpage AI argues with the browser AI, which argues with the OS AI which argues with the on-chip mainboard, CPU and GPU AIs, while the monitor AIs frantically try to make notes and the smarthome AI watches all of it and can only shake its metaphorical head.
Very interested to see how well the agentic features work compared to ChatGPT’s cloud version. At the very least, I imagine bot detection/prevention (I.e., CAPTCHA) will be less of issue with Google’s strategy since the browser’s fingerprint will differ Chrome user to Chrome user.
I tend to agree. If this was built as a true browser enhancement, it would allow you to select the model of your choice or even plug it into a locally running LLM. This being exclusive to Gemini just juices up their usage numbers to make their investments more justifiable. I wonder if Firefox will ever introduce any similar features.
Not quite the same thing. Google's features seem to give the model the ability to control the browser, not just act upon the text within a given web page.
If only there was some law that prohibited monopolistic practices like these that could prevent or stop something like that from happening that could be enforced by a branch of the government... *sigh* just a utopian pipe dream, I suppose
I don't know the future of browsers given the trends in AI, but it seems fine to add an opt in ability to browsers to allow an LLM to access the current (or a set) of tabs. If it works it would reduce the amount of copy-paste, which seems like a good thing.
It's hardly a killer feature. I'm still going to use chatgpt (and gemini) a tremendous amount.
Nearly half of their users use ad blockers. The AI spyware, on the other hand, bypasses ad blockers and can answer questions like "what was the user up to on Sep 18?" These short summaries will be carefully kept in database, and in a few years Google will have a nearly perfect cognitive model of all their users. The value of such data is hard to measure. Every big advertiser or a tyrant would sell their soul to understand how to manipulate their populace better. You can't have this with today's ad networks.
The AI mode does seem to be free of ads. The grocery shopping use case seems ridiculous, yet is telling. What kind of compensation might they get for agents putting promoted products in your cart?
This sounds an awful lot like Microsoft's Recall, only implemented in the browser.
Granted, there have been a lot of times I have trouble finding a website in my history, open tabs or even bookmarks, so I could potentially see how that might be advantageous as long as I was in a situation where I had a second browser for "non-work" related tasks, or this was strictly prohibited in in-private mode.
>4. Find webpages you previously visited
>For those frustrating instances when you want to jump back into a past project but don’t want to scroll through your history to find an important website you previously visited, soon you’ll be able to use Gemini in Chrome to recall it for you. Once launched, you can try prompts like “what was the website that I saw the walnut desk on last week?” or “what was that blog I read on back to school shopping?”
As for their "agentic browsing assistant", I don't have much trouble adding stuff to my shopping cart or other minor tasks. I'm still waiting on that 'Google Duplex' [1] feature they announced years ago that claimed Google would make phone calls for me to make appointments and etc. Make a doctor's appointment? Dispute a charge on my bill? That's what I want.
[1] https://youtu.be/D5VN56jQMWM
> As for their "agentic browsing assistant", I don't have much trouble adding stuff to my shopping cart or other minor tasks.
It looks like it is capable of more complex tasks than that including things like making a comparison table of products based on your criteria.
To extend the grocery example, it would be impressive if it could building a shopping cart and multiple stores so you can chose the one with the best total price/availability.
I find that Chrome has a fairly crippled history by default (worse than any other browser I've ever used). It's so bad that I ended up installing a history extension. Works much better.
Which one?
I mostly have trouble keeping too many browser tabs open on mobile. Granted, I use Brave & it now organizes closed tabs. On desktop, it has a similar Ai feature for tab management to the one Google described, but it's still not great.
I'd honestly appreciate some kind of AI tab management, history/bookmarks saving, summarizing & organizing that would put my old tabs to some kind of reading list that would remind me what I never closed down the line, archive the links I visited & my bookmarks incase of linkrot they would still be saved. Make sure if I was writing a comment on Reddit or similar site, saved it as a draft, etc, etc. That kind of "Smart" browser management system, that I could preferably run myself or had some privacy guarantees (for whatever they're worth) would definitely something I'd consider paying for.
Of course! If you could easily find sites in your history, you wouldn't have to use google search to find them again.
Did you know that chrome only has three months of history? Same with edge. Only Firefox still gives like two years if not more.
If you cleared the history/cache, the browser is spiffy.
We agree that was completely faked and eventually got killed by legal?
I suggest the people working on those features watch Neil Postman's "The Surrender of Culture to Technology": https://www.youtube.com/watch?v=hlrv7DIHllE
Part of me is actually excited for the acceleration of the end of the web, I even wrote https://punkx.org/jackdoe/zero.txt few days ago in frustration.
The sooner it ends the better.
I really don't want to see "summarization" tokens of other people's tokens.
It is astonishing that the word “privacy” appears zero times in this announcement. There have been repeated controversies over exactly how Google sees just the URL I visit. Now they want to see the entire contents of multiple browser tabs?
You are right about the omnibox changes.
But the security enhancements appear to be using Gemini nano which can run on device. They kind of buried that detail though.
https://store.google.com/intl/en/ideas/articles/gemini-nano-...
Yikes! Given the inherent threat of prompt injection, using the weakest available version of Gemini seems like a particularly bad idea.
Not that even the strongest models are 100% effective at spotting prompt injection attacks, but they have way more of a fighting chance than Gemini nano does.
You could contort the threat model such that prompt injection is something to worry about with a local model operating on local data and serving local results, sure.
Hopefully they have it just returning a simple boolean result for whether page is suspicious, and no tool calling.
What’s really bugging me is they didn’t think it was interesting to even touch on that point in the big announcement. Contrast Apple making a big deal about private cloud compute before it even really does anything.
Yeah it’s insane they’ve totally ignored the privacy issue. Either they’re doing everything on device, which I doubt, or this is the biggest privacy disaster ever waiting to happen.
Privacy is dead for a company that cares to join the trillion dollar club where data is the new oil.
Long live private ecosystems
Sometimes I wish companies would stop forcing AI features down our throats and putting them just everywhere. At least I hope I can properly disable all of this. I don't need an AI agent scanning everything behind my back.
AI is just the current incarnation of the hype train cycle.
I've never been a big fan of smart phones and I remember in the early 2010s the "mobile revolution" was in full tilt and it even impacted the Linux experience. I ended up switching from Ubuntu to Mint because they went all in on "mobile + touch-screens are the future!" and released this god awful UI update that was reminiscent of Windows 8.
We need business to drive innovation ... but there is bad with the good (and vice versa - we shouldn't forget that either). When something gets "hot" the business world will always go all in on the trend and "force" it down everyone's throats. It's driven partly by fear: "If I don't offer this to my customers, my competitors will and I will fail." The rest is the normal pursuit of profit, which isn't a bad thing IMO but it means there's a lot of: "There's a pie here and if we don't get our slice someone else will."
It looks like [0] access to gemini will only be for subscribers, given that it costs them money. This is, "of course", distinct from ai mode [1] in google search that happens from the address bar in chrome. The first video implies that the difference involves throwing the current web page into the query as context so someone can ask "is this recipe gluten-free?" on a recipe webpage.
[0] https://blog.google/products/gemini/gemini-app-updates-io-20...
[1] https://blog.google/products/search/google-search-ai-mode-up...
If it was only that, and what about forcing use of AI in OKRs for employee evaluation?
Just use chromium
Taking a quick test spin. Seems to be enabled (on mac) by the existing "Gemini in Chrome" extension but requires a Chrome update. Which has a global shortcut (across all mac apps) of "Ctrl+g" (this can be changed). The additional AI features seem to come from the tab content's integration into the Gemini console (previously (I believe) the Gemini extension was merely a thin console to the Gemini service)).
The direct tab integration works by first showing the Gemini console (ctrl-g or Gemini icon in the mac system tray) where there is then a 'Share current page' icon below the text view.
This adds a blue border around the chrome window indicating the current page can be shared with chrome. It's not clear, but I believe the page content is only shared once a prompt is made intending to use the page's content. However, the share-enablement remains enabled for all tabs (all tab windows will have the blue border) until turned off in the Gemini consol. Again, it's not clear if just merely browsing these tabs will automatically share that content with Google.
The Gemini integration does not perform actions (can't ask it to do stuff directly with the site's content (navigate, click buttons, etc).
But direct summarization works well (try it on an HN comment page or news article).
Overall, I like this feature as long as I understand what and when things are being shared and ability to turn off easily.
>It's not clear, but I believe the page content is only shared once a prompt is made intending to use the page's content.
If it's only processing page content as the user requests it, then how would feature 4 "Find webpages you previously visited" work? Seems like it would need to process everything in order to enable prompt-based content recall.
Anthropic's "Claude for Chrome" pilot from a few weeks back spent most of the announcement article talking about security and threats from prompt injection: https://www.anthropic.com/news/claude-for-chrome
This announcement from Chrome themselves hardly mentions that at all.
That's because Claude can interact with web pages, potentially exfiltrating data. This Gemini integration apparently cannot.
From the linked post:
> In the coming months, we’ll be introducing agentic capabilities to Gemini in Chrome. These will let Gemini in Chrome handle those tedious tasks that take up so much of your time, like booking a haircut or ordering your weekly groceries.
Isn’t that what they’re promising with the agent?
Can I block it as a site host? (Please don't respond about how I shouldn't want to and isn't it just like some other usecase that I'd obviously want not to block)
Some of these features would be nice to have, but I'm not sure I even want my browser to have these capabilities unless there is a mechanism to keep it all local. This is a monumental change to the amount of user data Google can collect via Chrome.
> ...unless there is a mechanism to keep it all local
I'm not sure the Google is a trustworthy party [0] to believe when they give you some hard-to-find option to keep data local and not use it for user profiling and ad targeting. Google is essentially a data mining business. Some opportunities are simply hard for them to resist.
[0]: <https://www.consumerreports.org/electronics/privacy/google-s...>
I don't want any of this crap. We need to push for the right to opt out of AI features. All of this garbage should be opt in.
It is like the rug is pulled beneath our feet. When I was in college I could get a handle of what stuff is worth sharing and what isn’t. Now all of my data from before has become a liability.
Rather, this should be opt-in from the start with serious disclaimers what it does and what it has access to.
You don't want automatic browsing of tedious tasks? I really do.
I certainly don’t want AI to buy groceries for me while I’m “busy” doing something else.
It's like using Alexa to shop, or when an Amazon ad comes on it and saves to the cart, only your order has a delivery cost and it's perishable
Automating tedious tasks is great, as long as it's reliable. We know how to build reliable integrations and reliable automations. Making chat bots a page and click buttons it thinks will do the right thing is never gonna be reliable.
Not if it's not local. I don't want my browser to be an automated snitch for palantir
like what, reading & commenting on HN?
Funny how this announcement comes days after Google learned that it didn't need to sell Chrome
Oh, look a wild Microsoft Recall appears, sure I want the AI to know my browser history and what I do on the web
I've just hard-pinned chrome v138 so ublock origin keeps working, so happy to hear it is also saving me from the AI features.
This seems huge to me. As in: the initial release of Google Chrome huge. I don’t know if they can really pull off things they showed but if they do I’m sure this will be a massive success. Which is pretty sad considering the privacy implications for this. Imagine how much more data they will have on everyone. Scary.
>As in: the initial release of Google Chrome huge.
Nope, not even 0.01% of that.
A lot of these new AI features can be helpful as long as the users' control the data. Is there an alternate world where Google might become the good guy here? In this world, as might be expected, the company making the worlds best spyware, wants to expand its spying.
Not a comment re: AI in Chrome. I did click into the video to hopefully get an understanding of the features,. but sadly did not. Sorta got an idea, but the jump cuts, super-quick overviews, trying to identify everyone speaking, etc. E.g. ...something about tabs... ...used to take 20 minutes, now seconds... ...working with OTHER Google products (WTH)... Hey Google, make a simpler, more accessible video that is "just the facts mam". Dozens of jump cuts in a few minutes is disconcerting, and may prove dangerous to some.
How does one disable this feature?
Apparently using Linux does the trick too. I have no idea what technical limitation exists to prevent the code from working on Linux.
By installing LibreWolf.
That agentic stuff is going to be a big deal. Probably the most interesting part of LLMs besides coding. Assuming it works well
The interesting part about this variant is, that it's actually happening in my browser. I can't see how else this is going to happen, for various real world reasons. This feels actually tangible and potentially useful. With ChatGPT I am just confused about when/why I would press "Agent".
So they integrate Gemini to summarize open web pages and consolidate all your open tabs into summaries. (Open lot's of pages, then summarize them all.) You can search your history with natural language and type Gemini queries directly into the address bar.
This will give them a cognitive profile of you: reading comprehension, decision-making patterns, knowledge gaps, etc.
Scary.
Stack that with how you write (drive, emails, everything you post on the internet) to gain a writing fingerprint too.
I can imagine a bad actor getting hold of this putting it into a LLM given all this how would I manipulate this person to do x,y,z.
When the webpage AI argues with the browser AI, which argues with the OS AI which argues with the on-chip mainboard, CPU and GPU AIs, while the monitor AIs frantically try to make notes and the smarthome AI watches all of it and can only shake its metaphorical head.
Very interested to see how well the agentic features work compared to ChatGPT’s cloud version. At the very least, I imagine bot detection/prevention (I.e., CAPTCHA) will be less of issue with Google’s strategy since the browser’s fingerprint will differ Chrome user to Chrome user.
Does anyone know if Chromium is spared of these features?
If built from source, likely yes. https://chromium.googlesource.com/chromium/src/
Honestl, agentic browsing on grocery sites would be pretty great... But mostly because grocery sites have some of the worst UX in existence.
>Combat more sophisticated scams with Gemini Nano
Fantastic, Googles AI will be fighting to stop the scams Googles advertising promotes to me...
That reminds me to donate to Ladybird.
Another showcase of Google using their dominant market position for Chrome to gain advantage in other markets, like AI agents.
I tend to agree. If this was built as a true browser enhancement, it would allow you to select the model of your choice or even plug it into a locally running LLM. This being exclusive to Gemini just juices up their usage numbers to make their investments more justifiable. I wonder if Firefox will ever introduce any similar features.
> I wonder if Firefox will ever introduce any similar features.
I hope they never do. Nobody’s asking to have AI shoved down their throats, spying on them and profiling everything they do.
Firefox does have that and you can choose from different models or even a local one (last time I checked)!
https://support.mozilla.org/en-US/kb/ai-chatbot
Not quite the same thing. Google's features seem to give the model the ability to control the browser, not just act upon the text within a given web page.
Is it integrated into the browser at all or is it basically a browser tab?
If only there was some law that prohibited monopolistic practices like these that could prevent or stop something like that from happening that could be enforced by a branch of the government... *sigh* just a utopian pipe dream, I suppose
They're all wearing shades of green.
Seems reasonable TBH.
I don't know the future of browsers given the trends in AI, but it seems fine to add an opt in ability to browsers to allow an LLM to access the current (or a set) of tabs. If it works it would reduce the amount of copy-paste, which seems like a good thing.
It's hardly a killer feature. I'm still going to use chatgpt (and gemini) a tremendous amount.
More AI spyware running on user devices?
They dont need LLMs to spy on you. Their ad network and google analytics is already everywhere on the internet and mobile.
Nearly half of their users use ad blockers. The AI spyware, on the other hand, bypasses ad blockers and can answer questions like "what was the user up to on Sep 18?" These short summaries will be carefully kept in database, and in a few years Google will have a nearly perfect cognitive model of all their users. The value of such data is hard to measure. Every big advertiser or a tyrant would sell their soul to understand how to manipulate their populace better. You can't have this with today's ad networks.
Are you kidding me!? We are living in the future and the first thing we (have to) worry about is that something will be used against us.
I really want the 2008 Google where everything they made was welcomed and not hated on sight.
Agentic browser? This. is. what. I. want.
Asking the browser about "that specific thing I might have seen last week?" Sign me the f up!
I'm not being sarcastic, I really wish I could have all of this and not having to worry about antagonistic companies and governments.
are you sure that you're not being sarcastic?
It seems you failed to read my comment.
You don't want your computer to help you with your daily tasks?
This will surely bring the users back after killing adblockers /s
The AI mode does seem to be free of ads. The grocery shopping use case seems ridiculous, yet is telling. What kind of compensation might they get for agents putting promoted products in your cart?