The current system works well, but it is not instant. As there is a delay between AdSense first seeing the code on a page and that page being crawled by Mediabot, it does mean there are times when AdSense is less aware of the content on a page, meaning that ads can be less targeted (and less valuable) when the URL is new to AdSense.
Enter the dynamic page
This can be particularly problematic for sites that employ dynamic URLs. The structure of some websites means that some URLs are totally unique to that page view and will never be seen again. This means that those URLs are always new to AdSense and even crawling them a few seconds later will not improve ad targeting as the URL will not get called again.
When AdSense does not understand the content of a page it partly relies on what it knows about a website as a whole. If an unknown URL called AdSense from a domain that largely talks about pets then it would be sensible to assume that that page is also likely to be pet related.
Whilst this works well for many sites it doesn’t help where a site covers a diverse range of topics, or where sub-themes might be more valuable.
An Example
Imagine that a book review website was sending its users an email that linked to daily, personalised book recommendations. This website might employ dynamic URLs so that users didn’t need to log in to view the recommendations, and those users would be likely to view each page once.
If ad units were placed on those pages then they would be new URLs to AdSense. AdSense might understand that the topic was likely to be about books, but not what that book was about.
How to improve AdSense targeting on dynamic pages
When an ad unit is called on a page for the first time, AdSense only has a single piece of information regarding that actual URL: The URL itself. They can apply what they know about the account, website, user and even the ad units, but if a publisher wants to influence the contents of contextual ads before the page is crawled then the URL is the only chance of passing that information to Google.
Does it work? Does Google target ads based on the URL?
To test this we created a deliberately bland, keyword-free, page that included an ad unit. We set this up so that it could be called from URLs containing any keyword that we chose. For example:
Example.com/kw/car_insurance/
Example.com/kw/london_hotels/
Example.com/kw/used_cars/
These URLs had never been used before and the keywords did on appear on the page. Viewing the pages from an incognito browser session ad units on those pages did indeed show ads that were relevant to the keyword in the URL.
It follows then, that if you are using “one use URLs” then adding a keyword into a URL string might help with ad targeting.
Example.com/6786568898fjhf78/ might then become
Example.com/fishing-books/6786568898fjhf78/
Does this mean that I can now pick the ads that appear on my pages?
Keywords in the URL works well on pages where AdSense has little other data to understand the context. Once the page is crawled the clues in the URL string have less effect.
We created a second test of a page called /car_insurance.php but this time replaced the bland, keyword free content with a few lines about London hotels. When we first visited the site ads were all, as expected, for car insurance. Watching the log files we spotted Mediabot arrive to check the page content just seconds later. Refreshing the page just 20 seconds after the first time the URL was ever seen by AdSense returned ads for London hotels.
In other words, the URL string is a useful signal for that first page load, but on-page content carries more weight once it is discovered.
What does this mean for publishers?
If you have to use single use URLs on AdSense pages then the recommendation would be to ensure that you have relevant keywords in the URL. This will help to improve the relevance of ads on the page.
Whether that keyword relevance is something that is of value to publishers using more traditional URL structures is hard to say. I certainly wouldn’t rule out the URL playing some part in determining a page’s relevance to AdSense terms in a similar way to how it is believed to work for Google Search.
