Bingbot: Discovering, Crawling, Extracting and Indexing (Fabrice Canel with Jason Barnard)
Fabrice Canel with Jason Barnard at The Bing Series
Fabrice canel talks to Jason Barnard about Bingbot.
If you would rather read than watch or listen, here is an article I wrote based on this conversation >>
This Episode Takes the Conversation a BIG step Further
This conversation is on a whole different planet. Fabrice is head of the entire discovery-crawling-extracting-indexing process. Think about how much that involves. And how important he and his team are to the process of getting your content to the top of the results.
You cannot hope to get your content into search results if it isn't found, crawled, extracted and indexed... and since he manages every single one of those steps, he is a person we really need to listen to.
Bingbot and Googlebot Function in Much the Same Way
Obviously they don't function exactly the same way down to the tiniest detail. But close enough ...
the process is exactly the same (discover, crawl, extract, index)the content they are indexing is exactly the samethe problems they face are exactly the samethe underlying technology they use is the same
So the details of exactly how they achieve each step will differ. But they are faced with the same environment and aim to do the same thing - index the web effectively. So, we can safely assume Google deals with the discovery-crawling-extracting-indexing process in a manner very, very close to Bing.
Just think about whatever industry you are in - details differ, but every competitor uses the same foundation. Easy to forget, but this is just another industry. So same here.
Google functions much the same way as Bing. And vice versa. Close enough for us not to need to worry too much about the differences.
Stunning Insights. I Learned sooooo Much.
The conversation with Frédéric Dubut that kicked off this series (this episode recorded at UnGagged) suddenly looks tame and unrevealing. A simple 'mise en bouche', as we say in French.
Listen and Learn
In short, as SEOs, we all depend on Fabrice and his team to an extent most of of us have probably will only start to grasp after watching the episode. This is the foundation of ranking in search. Everything else depends on this.
Fabrice is a truly lovely guy who wants to help you as a website manager... if only you'd help him help you. Here he tells you what he (and, presumably, his equivalent at Google) wants from you so that he can help you get your content to rank.
Help them overcome their problems, and you WILL be rewarded. Groovy !
Catch the rest of the Bing Series:
How Ranking Works at Bing - Frédéric Dubut, Senior Program Manager Lead, BingDiscovering, Crawling, Extracting and Indexing at Bing - Fabrice Canel Principal Program Manager, BingHow the Q&A / Featured Snippet Algorithm Works - (this episode) Ali Alvi, Principal Lead Program Manager AI Products, BingHow the Image and Video Algorithm Works - Meenaz Merchant, Principal Program Manager Lead, AI and Research,